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CHAPTER 1: 


Overview of ICILS 2018, its data, and 
implications for analysis 


Ekaterina Mikheeva and Sebastian 


1.1 Main objectives and scope 
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ICILS 2018 was based around research questions that focused on the following for CIL (in all 
countries) and CT (in countries where CT was also assessed): 
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Variations in CIL and CT within and across countries; 


reported proficiency in using computers; 
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CT. 


background) that are related to st 


The relationship between CIL and 


ICILS 2018 gathered data from abou 
schools from 12 countries and two 
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The following 12 countries and two benchmarking participants participated in ICILS 2018: 


Countries: 

e Chile (CIL) 
e Denmark (ClL and CT) 
e Finland (ClL and CT) 
e France (ClL and CT) 
e Germany (CIL and CT) 
e Italy (CIL) 
e Kazakhstan (CIL) 

e Korea, Republic of (CIL and CT) 
e Portugal (CIL and CT) 

e The United States (CIL and CT) 
e Uruguay (CIL) 


Benchmarking participants: 
e Moscow, Russian Federation (CIL) 
e North Rhine-Westphalia, Germany (CIL and CT)? 


1.2. The design in brief 


The ICILS 2018 international database (IDB) offers researchers and analysts a rich and 

innovative environment for examining student achievement in computer and information literacy 

and computational thinking in an international context. The database includes: 

e Extensive data on CIL and CT that enable in-depth study of the quality of education in terms 
of preparedness and learning outcomes; 


e Data for 12 countries and two benchmarking participants from around the world that can be 
used to develop an international perspective from which to examine educational practices and 
student outcomes in CIL education; 


e Data from eight countries and one benchmarking participant from around the world that can 
be used to assess educational practices and student skills in CT; 


e Student achievement in CIL and CT, linked to questionnaire information from students, schoo 
principals, and ICT coordinators, providing a source of policy-relevant contextual information 
onthe antecedents of achievement; 


e Scales for measuring students’ behavioral and emotional engagement with regard to ICT; and 


e Teacher questionnaire data that provide additional contextual information about the 
organization and culture of the sampled schools and information on general aspects of 
teaching pertaining to CIL and CT. 


The ICILS 2018 student target population comprised students in the grade that represents eight 
years of schooling, counting from Level 1 of the International Standard Classification of Education 


(ISCED), provided that the average age of students in this grade was at least 13.5 years at the time 
of the assessment. 


2 Data from North Rhine-Westphalia, Germany were included in the data for the whole of Germany. 
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“factor A influenced outcome B,’ and “variable A impacted on variable B” cannot and should not 


be established 
the study (Frai 
about causality. 


with ICILS 2018 data alone. The report containing the international results of 
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1.3.2 Estimation requirements 


Researchers familiar with population estimation in large-scale education survey data 
TIMSS, PIRLS, ICCS, or other IEA studies, will have little difficulty analyzing ICILS 20 
selves with the study’s conceptual foundation and its me 
CILS 2018 international data 
e data, this guide should provi 
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18 data once 
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technical information to enable them to conduct correct basic analysis. 

The three main design features of ICILS 2018 that need to be taken into account during any 

secondary analysis of the study’s data are: 

(1) The unequal selection probabilities of the sampling units that necessitate the use of sampling 
weights during computation of estimates; 

(2) The complex multistage cluster sample design that was implemented to ensure a balance 
between the research goals and cost-efficient operations; and 

(3) The rotated design of the CIL assessment test, wherein students completed only samples 
of the test items rather than the full set of test items. The CT assessment tests consists 
of two test modules which were rotated but administered to all students within countries 
participating in CT. 

Chapter 3 of this guide includes a brief account of the weights and variance estimation techniques 

intended for ICILS 2018, whereas Chapters 6, 7,and 13 of the ICILS 2018 technical report (Fraillon 

et al. 2020b) provide a more detailed description of the sample design and of the estimation and 


replication weights found in the international database. 
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theoretical and mathematical bases for multilevel analysis, readers should consult the existing 
literature on multilevel modeling. Section 3.2.1 of this user guide describes the use of weights in 
such models. Chapter 13 in the technical report (Fraillon et al. 2020b) includes a description of 
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(Fraillon et al. 2020a). 


linear modelling has been applied to results included in the international report 


When undertaking multilevel analysis of the ICILS 2018 data, users will need to take into account 
the structure of each participating education system. Although there are no major differences 
across the ICILS 2018 countries in how they defined a student for the purposes of the study, their 
definitions of what a school is (e.g., with respect to administrative units, multi-campus schools, 


buildings, tracks, 
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2. 


id vary across countries. The results of multilevel and variance 
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LLC 2019), HLM 6 (Raudenbusch et al. 2004), Mplus (Muthén & Muthén 2012), MLwiN (Rasbash 
et al. 2014), and SAS (SAS Institute 2013). 
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rates in the teacher questionnaire were below ICILS standards in France, 
Germany, Luxembourg, the United States, and Uruguay. These results are therefore presented 
separately in the ICILS 2018 international report (Fraillon et al. 2020a). Teacher data from 
these countries are at greater risk of non-response bias; these results should be interpreted 
with caution and cannot be compared directly with those from other countries. 


e Exclusion rates pertaining to the student population were above five percent in Denmark, 


e |taly assessed 


Kazakhstan and Portugal. The ICILS 2018 research team deemed this level of exclusion a 
significant reduction of target population coverage; researchers need to keep this feature in 
mind when interpreting results. 


its grade 8 students at the beginning of the school year and therefore the 


average age of sampled students was lower than 13.5 years at the time of assessment, which is 
below target group population age defined for the survey. As a consequence, CIL results from 
Italy are reported separately in the ICILS 2018 international report (Fraillon et al. 2020a). 


e |n Luxembourg, due to the relatively small number of schools and size of the target population, 
all eligible schools and students were included in the ICILS 2018 survey (census). 


Chapters 6 and 7 


of the ICILS 2018 technical report (Fraillon et al. 2020b) provide further details 


on participation and exclusion rates. 
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1.4 Contents of the user guide 


This user guide for the ICILS 2018 international database describes the content and format 
of the data contained in the database. In addition to this introduction, the guide includes the 
following chapters: 


e Chapter 2 describes the structure and content of the ICILS 2018 international database. 


e Chapter 3 introduces the use of weighting and variance estimation variables for analyzing the 
ICILS 2018 data. It also provides guidelines on comparing estimates. 


e Chapter 4 introduces the IEA IDB Analyzer software (IEA 2020) and, using this software 
in conjunction with SPSS (IBM Corporation 2016) and SAS (SAS Institute 2013), presents 
example analyses of the ICILS 2018 data. 


Three appendices also accompany this user guide. 


e Appendix A includes the international version of all international questionnaires administered 
inICILS 2018. These serve as areference guide to the questions asked and the variable names 
used to record the responses in the international database. 


e Appendix B provides details on all national adaptations applied to the national versions of 
all ICILS 2018 international questionnaires. When using the database, please refer to this 
supplement and check for any special adaptations to the background and perceptions variables 
that could potentially affect the results of analyses. 


e Appendix C describes the derived questionnaire variables, many of which were presented in 
tables in the ICILS 2018 international report (Fraillon et al. 2020a). 


CHAPTER 2: 
The ICILS 2018 international database files 


Ekaterina Mikheeva and Sebastian Meyer 


2.1 Overview 


The ICILS 2018 international database (IDB) contains student, teacher, and school data collected 
inthe 12 countries and two benchmarking participants that participated in the study. The database 
also includes data from the ICILS 2018 national contexts survey, providing information on the 
national contexts of computer and information literacy education for all participating countries. 


An operational code was assigned to each ICILS 2018 country in the international database (Table 
2.1). Only eight countries and one benchmarking participant took the optional CT modules; four 
of the ICILS 2018 countries also participated in ICILS 2013 (Table 2.1). 


For details on population coverage and exclusion rates for countries that participated in ICILS 
2018, please refer to chapter 6 of the ICILS 2018 technical report; for details on participation 
rates, please refer to chapter 7 of the ICILS 2018 technical report (Fraillon et al. 2020b). 


The database contains materials that provide additional information on its structure and content. 
This chapter describes the content of the database, and is divided into five major sections each 
covering the different file types and materials included in the database. 


Table 2.1: Countries participating in ICILS 2018 


Country Operational codes ICILS 2018 Participated 
modules taken in ICILS 2013 
Alpha-3 Numeric CIL Gil 
Chile CHL 152 ° ° 
Denmark? DNK 208 e ° e 
Finland FIN 246 ° ° 
France? FRA 250 ° ° 
Germany? DEU 276 e e e 
taly* ITA 380 ° 
azakhstan KAZ 398 J 
orea, Republic of OR 410 e e e 
Luxembourg? LUX 442 e e 
Portugal? PRT 620 e e 
Uruguay? URY 858 e 
United States*¢ USA 840 ° ° 
Benchmarking participants 
orth Rhine-Westphalia (Germany)*> DNW 276001 e e 
oscow (Russian Federation) RMO 6431 e 
Notes: 


Country met guidelines for sampling participation rates only after replacement schools were included. 
Country did not meet sample participation requirements for the student survey in ICILS 2013. 

Country did not meet sample participation requirements for the teacher survey. 

Country surveyed target grade in the first half of the school year. 

Country nearly met guidelines for sampling participation rates after replacement schools were included. 
Country did not meet sample participation requirements for the student survey. 
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2.2 ICILS 2018 database 


respective schools. The data files include the student respons 


are also contained in the international database. 


This chapter also describes the contents and format of the ICILS 


The ICILS 2018 database comprises data from all instruments administered to the students, the 
teachers teaching in the target grade, the school principals, and the ICT coordinators at the students’ 


es to the CIL and CT achievement 


items and the responses to the student, teacher, school, and ICT coordinator questionnaires. 
The files also contain the achievement scores estimated for participating students, as well as the 
background variables derived for reporting study findings in the ICILS 2018 international report 
(Fraillon et al. 2020a). National research coordinators’ responses to the national contexts survey 


2018 data files. These are provided 


in SPSS format (.sav) and SAS format (.sas7bdat), except for the national contexts survey data, 
which are only available in SPSS format (.sav). The files can be downloaded from the IEA Data 
Repository (at http://www.iea.n|/data-tools/repository). Data files are provided for each country 
that participated in ICILS 2018 where internationally comparable data are available.® 


The three types of ICILS 2018 data files in the database correspond to the three data levels 
established in ICILS 2018: school level, student level, and teacher level. Files of the same type 
include the same uniformly defined set of variables across countries. The file name identifies 
the type of data file and the country (Table 2.2). For example, BSGLUX212.sav is an SPSS file that 


contains Luxembourg’s ICILS 2018 target grade student data. 
data file for each participating country. 


Table 2.2: ICILS 2018 data file names 


Each file type contains a separate 


File names Descriptions 

BSGeee|2 Student achievement and questionnaire file 
BTGeee|2 Teacher questionnaire file 

BCGeee|2 School and ICT coordinator questionnaire file 
NCSICSI2 National contexts survey file 


Note: ¢ ee = three-character alphanumeric country code based on the ISO 3166 cod 


ing scheme (see Table 2.1) 


The SPSS files include full dictionary/meta information, that is, variable name, format (type, width, 
and decimals), label, value labels, missing values, and appropriately set measurement levels 
(nominal, ordinal, or scale). The dictionary information can be accessed through the SPSS “View 
> Variables” menu, or in output form through the “File > Display Data File Information” menu. 


SAS files include appropriate display formats and variable lab 
value labels in data files. 


All information related to the structure of the ICILS 2018 data 


els but do not permanently store 


files, as well as the source, format, 


descriptive labels, and response option codes for all variables, are contained in codebook files. Each 


type of data file in the database is accompanied by a codeboo 


k file in Excel format. The naming 


convention for codebook files follows the convention for the data files (see Table 2.2) except that 


the file extension is “xlsx”. 


Please note the SPSS data files are created in Unicode mode. 


However, when saving SPSS data 


files in Unicode encoding in code page mode, defined string widths are automatically tripled. These 


format changes will then cause problems when merging data 


with the IDB Analyzer (i.e., when 


merging the data in SPSS). Users should take this into account when saving data files in SPSS. 


6 Please refer to section 1.3.4 in chapter 1 for information about the constraints on data comparability. 
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2.2.1 Student data files (BSG) 


Studen 


constru 


were large task 


items, t 


Studen 


behaviors relev. 


The CT test modules 
CT). The CT test cons 


participati 
started wi 
and then u 
included u 


ng in 


ndert: 
n 

The student d 
ClLand CT (w 
featureanum 
variables, and 


report (see sect 
number (IDS1 


ique tasks 


ata files contain stud 
here app 
ber of identification variables, tracki 
derived variables that were used fo 


[UD). The IDSTU 


sth 


ts who participated in ICILS 2018 were also administered 
to answer questions related to t 


heir home backgr 
ant to CIL (and CT). 


followed the student questio 


the CT assessment. The m 
ook CT module 1. In addition to the 


that were assessed using mu 


ents’ responses 


icable) proficiency scores (pla 


r 


ion 2.4). Inthe student data files, eac 


D can be used to ide 


Item response code values 


Aseries of conventions also were adopted to code the data included in 


The values assigned 
choice items, numerical values from 1 


through 22 cor 
response 


The scoring, wh 
used aon 
specific criteria. 
on the nature o 
either O(n 
credit), 1 
a score of 
points on 


the ot 


The “missi 


S) is (are) i 


e-digit 


ocredit) or 1 ( 
partial credit), or 2 (full cred 


of the test i 
| through 2, 1 


to each 


respond to t 
ncluded 


ne respon 


in the value label. 


tomatic or hu 
Large tasks i 
ifestation of t 
k. In ClL test 


ether au 
scheme. 
The man 
f each tas 


nt 


her task. 


task. This code was only allocated when the entire sti 


area were left b 
nd checked wh 


he school datat 


2018 school da 


ether the response showed any devi 


.2.2 School data files (BCG) 


8 principal and ICT coordinator ques 


nse items, some were automatically scored computer-skil 
at were scored using analytic criteria. The stu 
responses to the multiple-choice questions and the scores assigned to the con 
he automatically scored skills items, and the large-task criteri 


isted of two test modules, which were both administe 
odules were rotated. This means 
th CT module 1 and then worked on CT 
tas 


+ 


to 


se options in individual items. 


man, of constructed-response 


he assessment criteria across t 
modules, some criteria allowed 
full credit) score points; others allowed for partial credit scoring as O (no 
it) score points. In CT test mod 
zero, one, or two points on one of the two tasks, and a score of zero, one, two, or three 


files contain responses from school pri 
the ICILS 201 
vestigative units can be performed, it is preferable to analyze school-leve 
f students or teachers. If users want to perform student- or teacher-level analyses with the ICILS 


ta, they will need to merge the school data files with the s 


nulti 


den 


a. 


aquestionna 


ound and their value beliefs, 


nnaire session (in countri 
red 


through 3, 1 through 4, O th 


ation from its initial state. 


ncipals and ICT coordi 


that some stu 
module 2, while others began with CT mod 
types used inthe ClL test modules, CT tasks 
iple criteria with multiple score categories. 


these questions. They also contain stud 
usible values). In addition, the student data files 
ng and timing variables, sampling and weig 


the ClLand Cl 


ts who participated in ICILS 2018 were administered two of five CIL test modules, each 
of which contained a series of tasks. Some of these tasks were n 
cted-respo 


ple-choice items, some were 
s tasks, and others 
t data files contain the actual 
structed-response 


ire that asked them 


attitudes, and 


es participating in 


to all students 
dent 
ule 2 


ents’ 


hting 


the analyses described in the international 
h student was assigned a unique identification 
ntify individual students within 


a country. 


| test data files. 


tem variables depend on the item format. For multiple- 


rough 11, or O 


For these items, the correct 


items and large-task criteria 
he ICILS 2018 test modules were all scored using task- 
he different tasks depended 
for dichotomous scoring as 


ules, students could receive 


ng” code (“9” in SPSS; “” in SAS) was used when a student made no attempt to answer a 
mulus, question stem, and question response 
ank by the student. The scoring system automatically allocated the “missing” code 


nators to the questions 
tionnaires. Although analysis with schools as 
variables as attributes 


tudent or teacher data 


ICILS 2018 USER GUIDE FOR THE INTERNATIONAL DATABASE 


files and to use the country and school identification variables to do so. Section 4.2 of this user 
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2.3. Records included 


The international database includes all records that satisfied the international sampling standards. 
Data from those respondents who either did not participate or did not pass adjudication because, 


for example, within-school participation was insufficient, were removed from the final database. 


More specifically, the database contains records for the following: 


e All participating schools: any school where the school principal responded to the principal 
questionnaire and/or the ICT coordinator responded to the ICT coordinator questionnaire 
has arecord in the school-level files. Participation in ICILS 2018 at school level is independent 
of participation at the student and/or teacher levels for the same school. 


e All participating teachers: any teacher who responded to the teacher questionnaire has a 
record in the teacher-level files, provided that at least 50 percent of the sampled teachers of 
that school participated in the study. 


e All participating students: any student who responded to at least one item of the student test 
has arecord in the student-level files, but only if at least 50 percent of the sampled students of 
that school took part in ICILS 2018. 


Consequently, the following records were excluded from the database: 
e Schools where both the principal andthe ICT coordinator did not respond to the questionnaire; 


e Students who could not or refused to participate or did not respond to a single item of the 
student test; 


e Students who experienced a technical failure of the electronic assessment system during test 
administration and were consequently unable to complete the assessment; 


e Students from those schools where less than 50 percent of the sampled students participated; 


e Teachers who did not respond to the questionnaire; 


e Teachers from those schools where less than 50 percent of the sampled teachers participated; 


e Students and/or teachers who were afterwards reported as not in scope, not eligible, or 
excluded; 


e Students or teachers who participated but were not part of the sample; and 


e Any other records that were considered unreliable, of undocumented origin, or otherwise in 
violation of accepted sampling and adjudication standards. 


Any additional data collected by countries to meet national requirements were also excluded from 
the international database. 


For further information on the ICILS 2018 participation and sampling adjudication requirements, 
refer to Chapter 5 of the ICILS 2018 technical report (Fraillon et al. 2020b). 
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The database contains the following information for each school that participated in the survey: 


e The identification variables for the country and school; 
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2.4.1 Identification variables 
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IDCNTRY indicates the country or participating education system that the data refers to as an 


up 


to six-digit numeric code based on the ISO 3166-1 classification (Table 2.1), with adaptations 
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reflecting the education systems participating. This variable should always be used as the first 
linking variable whenever files are linked within and across countries. 


CNTRY indicates the participant’s three-digit alphanumeric code, based on the ISO 3166-1 coding, 
with adaptations reflecting the education systems participating. 


IDSCHOOL is a four-digit identification code that uniquely identifies the participating schools 
within each country. The school codes are not unique across countries, but schools across countries 
can be uniquely identified by the combination of IDCNTRY and IDSCHOOL. 


IDSTUDis an eight-digit identification code that uniquely identifies each sampled student within a 
country. Students can be uniquely identified across countries using the combination of IDCNTRY 
and IDSTUD. The first four digits of IDSTUD contain the IDSCHOOL code for the student’s school, 
thus identifying a student within a school. 


EACH is a six-digit identification code that uniquely identifies the sampled teacher within a 
country. Teachers can be uniquely identified across countries using the combination of IDCNTRY 
IDTEACH. The first four digits of IDTEACH contain the IDSCHOOL code for the teacher’s 
sampled school. 


As for reasons of confidentiality, the identification variables for the student (IDSTUD), teacher 
(IDTEACH), and school (IDSCHOOL) were scrambled, they did not match the identifiers used 
during data collection. However, the structural link between the school and student/teacher level 
(the variable IDSCHOOL in the student and teacher files and the first four digits of any IDSTUD/ 
IDTEACH variable) was maintained for all countries. For each country, unique matching tables 
were created and made available to authorized individuals. 


Table 2.3: Location of identification variables in the |CILS 2018 international database 
Weight variables Data file 
BCG BSG BTG 

IDCNTRY ° ° ° 
CNTRY e e ° 
IDSCHOOL e e e 
IDSTUD ° 
IDTEACH ° 


Notes: BCG = school and ICT coordinator questionnaire file; BSG = student achievement and questionnaire file, BTG = teacher 
questionnaire file. 


2.4.2 Administration variables 


The international database includes several variables that provide additional information about 
survey administration, participation in the study, and other basic characteristics of respondents 
(Table 2.4). 


[LANGP indicates the language used in the principal questionnaire. The two-digit alphanumeric 
language codes are based on the ISO 639-1 standard (ISO [International Organization for 
Standardization] 2020). 


MODEA PrQ indicates the principal’s questionnaire mode. The variable is set to “1” if the 
questionnaire was completed online. It is set to “2” if it was completed on paper. 
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TADATE indicates the date (month/year) when the test 


language codes are based on the ISO 639-1 standard. 


MOD 


Table 2.4: Location of administration variables in the |CILS 2018 international database 


was administered to the student. 


TLANGC indicates the language used in the ICT coordinator questionnaire. The two-digit 
phanumeric language codes are based on the ISO 639-1 standard. 


ODEA CoQ indicates the ICT coordinator’s questionnaire mode. The variable is set to “1” if the 
tionnaire was completed online. It is set to “2” if it was completed on paper. 


DBOOK identifies the specific instrument version that was administered to each student via the 
ectronic ICILS 2018 assessment software. The instrument versions are given anu 
ranges from 11 through 30 for CIL test modules and from 31 through 70 for CT 


merical value 
test modules. 


GS indicates the language(s) in which the student test was written in a country and which 
student was actually required to use when working through the assessment. The two-digit 
alphanumeric language codes are based on the ISO 639-1 standard. 


uestionnaire. The two-digit alphanumeric 


EA_TcQ indicates the teacher’s questionnaire mode. The variable is set to “1” if the teacher 
completed the questionnaire online and “2” if he or she completed it on paper. 


Administration variables Data file 
BCG BSG BTG 
LANGP ° 
ODEA_PrQ ° 
LANGC ° 
ODEA_CoQ ° 
DBOOK ° 

LANGS ° 

TADATE ° 
LANGT ° 
ODEA_TcQ ° 


Notes: BCG = school and ICT coordinator questionnaire file; BSG = student achievement and questionnaire file, BTG = teacher 


questi 


onnaire file. 


2.4.3 Achievement item variables 


The names of the achievement item variables pertaining to the international test are based on an 
alphanumeric code (e.g., CLHO4AC). These codes consist of up to eight characters and adhere to 
the following rules: 


used to indicate the test modules: 


“B’ indicates the “Band Competition’module 


“G’ indicates the “Board Games” module 


The first character indicates the general study context. “C” stands for computer and information 
iteracy and “T” for computational thinking. 
The second character indicates the assessment cycle when the item was first used in ICILS. It 
therefore “1” for all items introduced as part of the ICILS 2013 assessment and “2” for items 
developed for ICILS 2018. 


The third character represents the test module the item belongs to. The following letters are 
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- “H” indicates the “Breathing” module 
— “R” indicates the “Recycling” module 
— “S” indicates the “School Trip” module 


- "A’ indicates the “Automated Bus” module 


— “F” indicates the “Farm Drone” module. 


e The fourth and fifth characters indicate the item number of the test module. 


e The sixth character is used for multipart items. “Z” is used for items not split into multiple 


parts. 


The seventh digit represents the original item type. “M” represents multiple-choice items; “O” 


stands for open-ended items. “A” represents items that were automatically scored, and “C” 
stands for items that were manually scored. “L’ is the indicator for items belonging to a large 


task. 


For example, C2RO3ZM is the third item from the student CIL test module “Recycling”. It is a 


multi 


2.4.4. Achievement test scores 


Cc Oo 


ple-choice item and was first developed for use in ICILS 2018. 


he ICILS 2018 research team produced the student CIL and CT achievement scales. Chapter 
11 of the ICILS 2018 technical report (Fraillon et al. 2020b) provides detailed descriptions of 


he ICILS 2018 scaling and the CIL and CT achievement scales, including their construction. The 
nternational database provides five separate estimates of each student’s score on that scale. 


he plausible values for the CIL and CT sca 


sing the IEA IDB Analyzer, described in detail 


The five achievement score variable, i.e. plausib 
a 


nd PV1CT to PV5CT for the CT scale. 


2.4.5 Questionnaire variables 


[hese are contained in the student file. The variabi 
as “plausible values,” encapsulates the uncertai 


nty 


inherent in the scale estimation process. 


+ 


y between the five estimated scores, known 


es are the best available measures of student 


achievement on these scales in the international d 
utcome measure in any study of student achievement. Plausible values can be readily analyzed 
in chapter 4 of this user guide. 


e value, names are PV1ICIL to PV5VIL for the CIL 


atabase and should therefore be used as the 


The questionnaire variable names consist of a 6- to 8-character string (e.g., IS2GO4A). The 
following rules are applied in naming the variables of the international and national instruments: 


e The first character indicates the reference level. The letter “l” is used for variables that are 


e The second character indicates the type of res 
from school principals, while the letter “|” is used for |CT-coordinator data. The letter “T” is 
used for teacher data. The letter “S” is used for 


e The third character indicates the study cycle. N 


administered on an international level. 


cycle of the IEA study. 


e The fourth character is used to indicate the 


e The fifth and sixth characters indicate the ques 


general contexts. 


pondent. The letter “C” is used to identify data 


student data. 
umber “2” identifies ICILS 2018 as the second 


context of the variable. The letter “G” is used for 


tion number. 


e The seventh and eighth characters represent optional digits for multipart items, and optional 


digits for multipart subitems, respectively. 
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The values assigned to each of the questionnaire variables depend on th 


and the number of options availab 
used that correspond to the respon 


se options available. The nu 


e questionnaire item format 


le. For categorical questions, sequential numerical values are 
mbers correspond to the sequence 


of appearance of the response options. For example, the first response option is represented with 


a 1, the second response option with a 2, a 


stu 


The raw informa 


and inconsistent 
necessary. 
example, 


ple 


With respect to the internati 
information coded in each vari 
were reflected appropriately in all concerned va 
comparable were removed from the database. 


dents in a school,’ are coded 


the removal of implau 


tion collected by th 


with the actual number given 


sible responses, for all countr 
collection, capturing, processing, editing, weighting, and adjudi 


e questionnaires underwent exten 
cleaning, and editing. Out-of-range values, questions determi 
or implausible co 


nd so on. Open-ended questions, such as “number of 


ase consult chapters 7 and 10 of the ICILS 2018 technical report ( 


onal database, the data-cleaning process at 


as aresponse. 


sive processing, inspection, 


ning the flow of the questionnaire, 
mbinations of responses were ins 
To address residual inconsistencies, |CILS 2018 imposed certain automatic edits, for 
ies. For further information on data 
cation of the international database, 


pected and cleaned where 


Fraillon et al. 2020b). 


EA ensured that 


able would be internationally comparable. National adaptations 


their eventual handling, consult Appendix B of this user guide. 


2.4.6 Indices, ratios, and ind 


icators derived 


Several questions asking about various as 


ICI 
ind 


LS 2018 questionnaires. In 


ividual items in order to create a derived variable that provided am 


these cases, 


of the construct of interest than the individ 


The international database con 


tains scale in 


riables, and questions that were not internationally 
For more information on national adaptations and 


from the questionnaire data 


pects of a single construct appear frequently in the 
the ICILS research team combined responses to the 


ual variables could on thei 


dices derived from scaling 


ore comprehensive picture 
rown. 


of items, a process typically 


achieved by using item response modeling of dichotomous or Likert-type items. Questionnaire 


SCca 
an 


Appendix C of this user guide 
international database. For fur 
items, 


2.4.7 


nthe ICILS 2018 questionnaires, typically sets of items reflecting a number of different aspects 


les derived from weighted 


ther indices t 


tionnaire va 


hat were derived by simp 
riables. 


ikelihood estimates (logits) present values on a continuum with 
ICILS average of 50 and a standard deviation of 10 (for equally 
The database also contains o 
transformation of original ques 


weighted national samples). 


le recoding or arithmetical 


provides a description of all derived variables included in the 
ther information about the scaling procedure for questionnaire 
please refer to Chapter 12 of the ICILS 2018 technical report (Fraillon et al. 2020b). 


Summary scales and derived variables from the questionnaires 


were used to measure a single construct. In these cases, responses to the individual items were 


combined to create a derived 
construct of interest than relying on individ 


Inthe ICILS 2018 reports, a scale is a speci 


variable wh 


ual item responses. 


ich provided a more comprehensive picture of the 


al type of derived variable that assigns a score value 


to students on the basis of their responses to the component variables. In ICILS 2018 new scales 


were typically c 


alculated as IRT weighted 


ikelihood estimation (WL 


E) scores with mean of 50 


and standard deviation of 10 for equally weighted countries. Scales based on (unmodified) item 


sets already included in the ICILS 2013 qu 


estionnaire were equated and their scale scores are 


comparable with the scales established in the previous survey; in these cases, the metric reflects 
a mean of 50 and a standard deviation in the pooled ICILS 2013 sample giving equal weights to 
each participating country. For student, teacher, and school questionnaire scaling, we only included 
records in the scale calculation if there were data for at least two of the corresponding indicator 


Var 


iables. 
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In addition to the scale indices, the ICILS 2018 international database also contains other 
(simple) indices that were derived by simple recoding or arithmetical transformation of original 
questionnaire variables. 


Appendix C to this user guide provides a description of all derived variables (scale scores and 
indices) included in the international database. For further information about the scaling procedure 
for questionnaire items refer to Chapter 11 of the ICILS 2018 technical report (Fraillon et al. 
2020b). 


2.4.8 Weighting and variance estimation variables 


To allow for calculation of the population estimates and correct jackknife variance estimates, the 
data files provide sampling and weighting variables. Further details about weighting and variance 
estimation are provided in Chapter 3 of this user guide. 


Each record in the ICILS 2018 international database contains one or more variables that reflect 
the record’s selection probabilities (or base weights) and nonresponse adjustment(s). The last 
character of the variable name indicates the data type (student = S, teacher = T, school = C). The 
weights and weighting factors differ depending on the data type. The only value identical in all three 
types of datasets is the value for the school base weight (variable WGTFAC1). This is because the 
school sampling comprised universally the first sampling stage and is therefore independent of 
data type. Each data file contains an estimation or final weight variable. Each such variable starts 
with the letters “TOT” (i.e., the product of all other weight variables) and must be used for single- 
level analyses. 


The ICILS 2018 international database contains 12 weight variables (Table 2.5). 


WGTFAC1 is the school base weight. It corresponds to the inverse of the selection probability 
of the school. 


TOTWGITC is the final school weight for schools. It is computed as the product of WGTFAC1 and 
WGTADJ1C. The final school weight for schools must be applied when analyzing the data coming 
from the school questionnaire. 


WGTADJ1C is the school weight adjustment for schools. It accounts for the non-returned school 
questionnaires. 


TOTWGTS is the final student weight. It is computed as the product of WGTFAC1, WGTADJ1S, 
WGTFACSS, and WGTADJS3S. The final student weight must be applied when analyzing the 
students’ data. 


WGTADJ1S is the school weight adjustment for students. It accounts for the sampled schools that 
did not participate in the student survey. The adjustment is done within explicit strata. 


WGTFACSS is the student base weight. It corresponds to the inverse of the selection probability 
of the student. 


WGTADJ3S is the student weight adjustment. It accounts for the sampled students that did not 
participate in the student survey. The adjustment is done within schools. 


TOTWGTT is the final teacher weight. It is computed as the product of the WGTFAC1, WGTADJ1T, 
WGTFAC2T, WGTADJ2T, and WGTADJS3T. The final teacher weight must be applied when 
analyzing the teacher's data. 


WGTADJ1T is the school weight adjustment for teachers. It accounts for the sampled schools that 
did not participate in the teacher survey. The adjustment is done within explicit strata. 


WGTFAC2T is the teacher weight factor. It corresponds to the inverse of the selection probability 
of the teacher within the school. 
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WGTADJ2T is the teacher weight adjustment. It accounts for sampled teachers that did not 


participate | 


WGTFAC3 
one school. 


Table 2.5: 


Location of weighting variables in the ICI 


nthe teacher questionnaire. The adjustment is done within schools. 


LS 2018 international database 


is the teacher multiplicity adjustment. It accounts for teachers teaching in more than 


Weighting variables 


Data file 


BCG 


BSG BTG 


WGTADJ 


WGTFAC3 


Notes: BCG = school and ICT coordinator questionnaire file; BSG = student achievement and questionnaire file, BTG = teacher 
questionnaire file. 


A variance estimation method that considers the structure of the data is the jackknife repeated 


replication (JRR) me 


the implementation of this method (ie., “jackkni 


strongly encourage d 
the data structure of 


with the respective variables. 


thod. The ICILS 2018 internationa 


database contains variables that support 


fe zone’ “jackknife replicate?’ “replicate weights”); we 


atabase users to use them. As the IEAIDB Analyzer automatically recognizes 


ICILS 2018, it reports correct standard errors for all estimates using JRR 


Several variance estimation variables (or "jackknife variables") are included in the ICILS 2018 
replicate weights are computed "on-the-fly" within 
to conduct analyses without using the |DB Analyzer, 


internat 


these va 


between 


or its we 


for thes 


between 


JKREPT 


JKZONES indicates to which sampling zone th 
JKREPSca 
The variables SRWGT1 to SRWGT75 indica 
tudent survey. 


JKZONET indicates to which sampling zone th 


Ca 


or its we 


ional database (Table 2.6). The actual 
the IDB Analyzer, but, as researchers may wish 
riables are also presented within the 


1 and 75. This variable is used to esti 


ght doubled when estimating sampli 


1 and 75. This variable is used to esti 


data. 
e student belongs. The values of JKZON 


mate sampling errors when analyzing st 


ng errors. 


e teacher belongs. The values of JKZON 


for the teacher questionnaire. 


ng errors. 


ES can vary 
udent data. 


n take the values O or 1. This variable indicates whether the student should be deleted 


te the jackknife replicate weights variables (1-75) 


ET canvary 


mate sampling errors when analyzing teacher data. 


n take the values O or 1. This variable indicates whether the teacher should be deleted 
ight doubled when estimating sampli 


The variables TRWGT1 to TRWGT75 indicate the jackknife replicate weights variables (1-75) 
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JKZONEC indicates to which sampling zone the school belongs. The values of JKZONEC can vary 
between 1 and 75. This variable is used to estimate sampling errors when analyzing school data. 


JKREPC can take the values O or 1. It indicates if the school should be deleted or its weight doubled 
when estimating sampling errors. 


The variables CRWGT1 to CRWGT75 indicate the jackknife replicate weights variables (1-75) 
for the school survey. 


Table 2.6: Location of variance estimation variables in the ICILS 2018 international database 


Variance estimation 
variables 


Data file 


BCG 


BSG 


BTG 


JKZO 


NEC 


JKREPC 


CRWGT1 to CRWGT75 


JKZO 


ES 


JKREPS 


SRWGT1 to SRWGT75 


JKZO 


ET 


JKREPT 


RWGT1 to TRWGT75 


Notes: BCG = school and ICT coordinator questionnaire file; BSG = student achievement and questionnaire file, BTG = teacher 


question 


naire file. 


2.4.9 Database creation variables 
umber of the ICILS 2018 international database and the date of its 


Information about the version n 
creation by IEA is contained i 


VERSI 


ON indicates the syst 


SCOP 
code “ 


IEADATE specifies the date when | 


3” indicates that they are 


2.5 Coding of missing data 


EA produced the data file. 


n the database creation variables. They are included in all data files. 


em of database version numbers was used throughout the data 
processing process. The version number of the ICILS 2018 final database is “5.0” or higher. 


E identifies the scope of the data filles. Code “2” indicates the filles are restricted use files and 
public use files. 


A subset of the values for each variable type was reserved for specific codes related to different 
categories of missing data. We recommend that the user reads the following section with particular 
care since the way in which these missing codes are used may have major consequences for analyses. 


2.5.1 Omitted response codes (SPSS: 9, 99, 999, ...; SAS:.) 


“Omitted” response codes are used for questions or items that astudent, teacher, or school principal 
should have answered but did not, i.e. an omitted response code is given when an item is left blank. 
The length of the omitted response code given to a variable in the SPSS data files depends on the 
number of characters needed to represent the variable. For example, the omitted code for a one- 
digit variable is “9 whereas the omitted code for three-digit variables would be “999”, 
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2.5.2 Not administered response codes (SPSS: 8, 98, 998, ...; SAS: .A) 


S 


pecific codes were given to items that were “not administered” to distinguish these from data 


that were missing due to non-response. The not administered code was used in the following 
cases: 


ao 4 


Z. 


A 


2 


CIL test item was not assigned to the student: All students participating in ICILS 2018 CIL test 
received only two of the five CIL test modules. All variables corresponding to items that were 
not part of the modules assigned to a student were coded as “not administered”. 

CT test item was not assigned to the student: The ICILS 2018 IDB includes CT-related variables 
even for countries that did not participate in the CT module. All variables corresponding to 
items that were not part of the assigned CT test module were coded as “not administered”. 


Student was absent from test session: When a student did not attend a particular testing session, 
for example because of sickness, all variables relevant to that session were coded as “not 
administered”. 


The achievement item was not displayed to the student due to a technical failure of the electronic 
assessment system: Ifthe assessment system failed during the assessment, all variables following 
the last item presented to a student when the failure occurred (i.e., assuming there was still 
time left to complete the corresponding test module) were coded as “not administered” (see 
chapter 11 in the ICILS 2018 technical report; Fraillon et al. 2020b). 


Question or item deleted or mistranslated: A question or item identified during translation 
verification or item review as having a translation error, such that the nature of the question 
was altered, or as having poor psychometric properties, was coded as “not administered” if it 
could not be recoded to match as closely as possible the international version. 


A questionnaire was returned empty, was not returned, or was lost: All variables referring to that 
instrument and any derived variables were coded as “not administered.” 

A country chose, for cultural reasons, not to administer (include) a certain question in its national 
questionnaire: The variables corresponding to the removed question were coded as “not 
administered.’ Appendix B of this user guide provides details on the national adaptations. 


he length of the not administered response code in the SPSS data files depends on the number 
f characters needed to represent the variable. For example, the not administered code for a one- 
git variable is “8,” whereas the not administered code for three-digit variables would be “998”. 


5.3 Not reached response codes (SPSS: 7; SAS: .R) 


n item was considered “not reached’ in the achievement data files when the item itself and the item 


preceding it were not answered and when (1) no other items were completed in the remainder of 
the test module, and (2) no technical failure of the electronic student assessment system occurred. 


.5.4 Logically not applicable response codes (SPSS: 6, 96, 996, ...., SAS: .B) 


“Logically not applicable’ response codes were used for the questionnaire items for which responses 
were dependent on a filter question. If the filter question was answered such that the following 
questions would not apply any follow-up question has been coded as “logically not applicable”. 


T 


he length of the logically not applicable response code in the SPSS data files depends on the 


number of characters needed to represent the variable. For example, the logically not applicable 
code for a one-digit variable is “6.” whereas the logically not applicable code for three-digit variables 
would be “996”, 
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2.6 Codebook files 


All information related to the structure of the ICILS 2018 data files, as well as the source, format, 


descriptive labels, 
Excel format. 


Inthe codebook file, t 


an 


d response option codes for all varia 


here is atab for each appropriate data t 


bles, is contained in a codebook file in 


fle type inthe ICILS 2018 international 


database. These tabs describe the contents and structure of the respective data file. Important 


codeboo 
Scheme 
Scheme 


Detailed)’ 


Ww 


hich lists the acceptable responses a 
Detailed,’ which lists all applicable missing codes in SPSS and SAS. 


2.7. Twoversions of the ICILS 2018 international database 


fields include “Label.” which contains extended textual information for all variables, “Value 
lowed for each variable, and “Missing 


To protect the confidentiality of the study respondents, ICILS 2018 applied certain disclosure- 


avoidance measu 


res at the international level. These measures were consis 


tent across all 


countries. These measures were implemented for all data versions and exports of the database 


that participating cou 


prevented by app 


variables and jackknif 


were needed only 
were identified as 


The ICILS 2018 in 


yin 


during fi 
highly id 


ternatio 


restricted use file ( 


RUF). Th 


the pu 


g international disclosu 
e zone information. Furthermore some of 


eld operations and 


entifying were suppressed or categori 


nal database is ava 


ntries and public users can access. Indirect identification of i 
re risk edits, such as scrambling o 


ndividuals was 
f identification 
the personal data variables that 


data processing were removed, and variables that 


ilable in two versio 


e public use version 


Repository (https://www.iea.n|/data-tools/repos 
or categorized from 
information or allow re-identification. Users wi 


b 


results with this version of 
extended version for scienti 
their analyses should contac 
to the restricted use version 


ic use version in ord 


the ICILS 2018 inte 


is available for imm 
itory). Anumber 


zed. 


nsia 
ediat 


| be able to rep 


rnational database. T 
fic use. Users who require any of the rem 
t the IEA through its Data Repository to ob 
of the ICILS 2018 international database. 


icate 


of variables have been r 
er to minimize the ris 


public use fi 
e access fromthe | 


of disclosing con 


he restricted use 


tain permission and 


Some variables in the restricted and the public use versions of the ICILS 2018 intern 
database have been scrambled, categorized, or removed for differing reasons (see Tables 2.7, 2.8 
and 2.9). More details for all of these variables are available in the codebook files (see section 2.6). 


le (PUF) anda 
EA 
emoved 


Data 


dential 


all published ICILS 2018 
file is an 
oved variables to conduct 


access 


ational 
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Table 2.7: Disclosure risk edits for sampling, identification, and tracking variables 


Variables Description Location of RUF PUF 
data files 
DSCHOOL/ ID variables All files Scrambled Scrambled 
DSTUD/|IDTEACH 
JKZONEC/T/S Jackknife zone BCG, BTG, BSG Scrambled Scrambled 
DSTRATE Explicit stratum code BCG, BTG, BSG Suppressed Suppressed 
DSTRAT| Implicit stratum code BCG, BTG, BSG Suppressed Suppressed 
ODEA_CoQ/ Administration mode school BCG Included Suppressed 
ODEA_PrQ 
ODEA_TcQ Administration mode teacher BTG Included Suppressed 
TEACHERS umber of teachers in target grade BCG Suppressed Suppressed 
BIRTHY_T Teachers’ year of birth from tracking BTG Suppressed Suppressed 
orms 
BIRTHY_S, Students’ year/month of birth from BSG Suppressed Suppressed 
BIRTHM_S tracking forms 
TADATE Date of testing BSG Included Suppressed 
Notes: RUF = restricted use file; PUF = public use file; BCG = school and ICT coordinator questionnaire file; BSG = student achievement and questionnaire file, 


BTG = teacher questionnaire file. 


Table 2.8: Disclosure risk edits for school questionnaire variables 


Variables Description Location of RUF RUE 
data files 
P2GO08A Public or private school BCG cluded Suppressed 
P_PRIV Public or private school - derived BCG cluded Suppressed 
P2GO03A/IP2G03B Total enrollment BCG cluded Suppressed 
P_NUMSTD Total enrollment - derived BCG cluded Categorized 
P2G04A/IP2G04B Enrollment <target grade> BCG cluded Suppressed 
P_NUMTAR Enrollment <target grade> - derived BCG cluded Categorized 
P2GO6A/IP2G06B Total number of full-time and part-time BCG cluded Suppressed 
eachers 
P_NUMTCH Total number of full-time and part-time BCG cluded Categorized 
eachers - derived 
Notes: RUF = restricted use file; PUF = public use file; BCG = school and ICT coordinator questionnaire file; BSG = student achievement and questionnaire file, 


BTG = teacher questionnaire file. 


Table 2.9: Disclosure risk edits for student questionnaire variables 


Variables Description Location of RUF PUF 
data files 

IS2GO01A, IS2GO1B Students’ date of birth (month, year) BSG Suppressed Suppressed 
IS2GO7A, IS2GQO7B, Parent / guardian 1’s job (open-ended) BSG Suppressed Suppressed 
IS2GOB8A, IS2GO08B 

IS2G11A, IS2G11B, Parent / guardian 2’s job (open-ended) BSG Suppressed Suppressed 
IS2G12A, IS2G12B 

Notes: RUF = restricted use file; PUF = public use file; BCG = school and ICT coordinator questionnaire file; BSG = student achievement and questionnaire file, 


BTG = teacher questionnaire file. 


CHAPTER 3: 


Weights and variance estimation for 
ICILS 2018 


Sabine Tieck and Sabine Meinck 


3.1 Overview 


This chapter provides an introduction to the use of weight and variance estimation variables in the 
ICILS 2018 student, teacher, and school data analyses, and its content is largely based on chapter 
3 of the ICILS 2013 User Guide (Jung & Carstens 2015). The examples outlined here demonstrate 
the importance of using appropriate weight variables and variance estimation techniques in order to 
achieve acorrect estimation of population parameters and calculate standard errors that correctly 
reflect the uncertainty of these parameters. It is important to account for the ICILS survey design 
if researchers are to draw correct conclusions about the population under study. This chapter also 
includes a discussion of constraints for specific analysis types (e.g., when simultaneously using 
data from different sources). 


3.2 Sampling weights 


All data in the ICILS 2018 international database were derived from randomly drawn samples of 
schools, students, and teachers. In order to arrive to unbiased estimates of the target population 
under study, database users, and analysts must take into account the complex nature of the sampling 
design implemented in each ICILS education system. Chapter 6 of the ICILS 2018 technical report 
(Fraillon et al. 2020b) provides details about ICILS 2018 sampling design. 


This complex design resulted in varying selection probabilities for sampled schools, students, and 
teachers. Another consideration arising out of this design is that the varying nonparticipation 
patterns of schools among strata and of students/teachers within participating schools can lead 
to biased estimates. All units participating in ICILS 2018 have sampling weights that consider 
these two design features, permitting an unbiased estimation of population parameters. Chapter 
7 of the ICILS 2018 technical report (Fraillon et al. 2020b) elucidates on the weighting and 
nonparticipation adjustments. 


Sampling weights were calculated for the ICILS 2018 student, teacher, and school populations to 
enable correct analyses of the data. 


3.2.1 Student weight variables 
Five student weight variables are included in the ICILS 2018 international database (Table 3.1). 


Table 3.1: Weight variables in student achievement and questionnaire file (BSG) 


Variable Description 

TOTWGTS Final student weight 

WGTFAC1 School base weight 

WGTADJ1S School weight adjustment - student study 
WGTFAC3S Student base weight 

WGTADJ3S Student weight adjustment 


Note: For a full description of the weight variables, see section 2.4.8. 
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3.2.2 Teacher weight variables 
Six teacher weight variables are included in the ICILS 2018 international database (Table 3.2). 


Table 3.2: Weight variables 


in teacher questionnaire file (BTG) 


Variable Description 

TOTWGTT Final teacher weight 

WGTFAC1 School base weight 

WGTADJ1T School weight adjustment - teacher study 
WGTFAC2 Teacher base weight 

WGTADJ2T Teacher weight adjustment 

WGTFAC3 Teacher multiplicity adjustment 


Note: For a full description of the weight variables, see section 2.4.8. 


3.2.3 School weight variables 


Three school weight var 


Table 3.3: Weight variables 


iables are included the ICILS 2018 international database (Table 3.3). 


in school and ICT coordinator questionnaire file (BCG) 


Variable Description 

TOTWGTC Final school weight 

WAGFAC 1. School base weight 

WGTADJ1C School weight adjustment - school study 


Note: For a full description of the weight variables, see section 2.4.8. 


3.2.4 Selecting the appropriate weight variable 


Researchers analyzing ICILS 2018 data must use sampling weights that consider the study's 
complex sample design in order to obtain unbiased population estimates. The choice of correct 
sampling weights will depend on the type of data used and the level of analysis. Section 2.4.8 of 
this user guide lists and describes all weight variables in the international database. It also provides 
the variable labels and source files. 
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LS students from disadvantaged backgrounds did not respond 
y estimation of ClL average scores controlling 


for these variables would inevitably lead to biased resu 


background (Fraillon et al. 2019). Multiple imputation methods offer a possible solution for dealing 


with missing data issues. 
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e At level 2 (school level), a “school weight” should be used for analysis. During analysis of 
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Country Student cluster size 
Minimum Maximum Mean Standard error 
Chile 3 24 17.4 40 
Denmark 5 24 16.8 3.0 
Finland ‘I 21 17.7 3.0 
France 11 20 18.9 2 
Germany 5 20 lees) 2.8 
taly 8 22 18.7 7 
azakhstan 4 24 18.4 3.9 
orea 7 20 19:2 8 
Luxembourg 23 333 142.1 67.6 
oscow, Russian Federation ‘i 20 19.0 5 
orth Rhine-Westphalia, 10 20 18.3 8 
Germany 
Portugal 7 24 16.1 31 
United States 4 36 25.8 3.9 
Uruguay 5 22 15:7 3.1 
If multilevel analyses are done using the entire national sample, the sample size should generally 
be sufficiently large for conducting this type of analysis. However, if the analysis is undertaken 
only for subgroups of schools, researchers should ensure that there are no fewer than 30 schools 


within each subgroup. 


For the majority of participating countries, conducting multilevel analysis with teacher data is 
unlikely to 
school is close to 10 (Table 3.5); hence, a significant number of schools have smaller cluster sizes. 
In this instance, single-level analysis may be preferable in order to obtain more reliable results. 


result in precise level 


1 estimates. The average number of responding teachers per 


8 Given that the school sample size of 38 cases at level 2 for Luxembourg provided only limited statistical power, data 
from this country were not included in the multilevel analysis reported for ICILS 2018 (Fraillon et al. 2020a). 
9 See Meinck and Vandenplas (2012) for details and an extensive literature review on the topic. 
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Table 3.5: Number of participating teachers per participating school (teacher cluster size) 


Country Teacher cluster size 
Minimum Maximum Mean Standard error 
Chile 3 9 GT 3.6 
Denmark 2 8 8.1 3.5 
Finland 3 8 3.0 27 
France 6 9 2.0 2.5 
Germany 2 9 2.8 2.6 
taly 3 9 2.0 3.6 
azakhstan 7 9 43 2.0 
orea 5 9 44 29 
Luxembourg lal 25 76 43 
oscow, Russian Federation 7 8 49 0.9 
orth Rhine-Westphalia, 5 8 we 9 
Germany 
Portugal 5 9 3:5 23 
United States 2 20 24 5.6 
Uruguay 6 7 0.9 2.5 
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e Explicit stratification and disproportional sample allocation was commonly used. This practice 


would lead to further variation in school selection probabilities. 


e Nonresponse patterns vary in accordance with nonresponse adjustment cells (i.e., strata or 
schools). For instance, individual student weights in schools with a response rate of just over 
50 percent" would be almost twice as large as those from schools where all sampled students 


participated. 


10 All sampling units have similar estimation weights. This is achieved by assigning low selection probabilities to small 
schools but high selection probabilities to students within small schools and, vice versa, high selection probabilities to 
large schools but low selection probabilities to students within large schools. The product of the two base weights is 
then similar for all students. See Meinck (2015) for further reading on this matter. 


11 Note that ICILS 2018 considered schools with response rates below 50 percent as refusals and hence assigned them a 


weight of zero. 
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Circumstances such as these make using weights in all |CILS 2018 data analysis essential if biased 
results are to be avoided. In other words, not using weights in data analysis can lead to severely 
biased results. 


Our next example illustrates this importance. Imagine a researcher is interested in ascertaining 
the CIL average in France (variables PV1CIL-PV5CIL in the BSG file) and is using (e.g., in SPSS) 
unweighted data. The mean of each plausible value is calculated first, followed by the average of 
the five values. The average score would turn out to be 496.36 (Figure 3.1). 


Figure 3.1: Example of unweighted analysis in SPSS 


N Minimum Maximum Mean Std.error | Std. deviation 
Computer and information literacy - Ist PV 2940 93.50 742.16 496.8793 1.46355 79.35634 
Computer and information literacy - 2nd PV 2940 162.73 719.86 [ 495.5944 \ 1.47520 79.98770 
Computer and information literacy - 3rd PV 2940 14.72 736.11 196.3210 1.48353 80.43958 
Computer and information literacy - 4th PV 2940 161.27 727.93 196.4655 1.48351 80.43830 
Computer and information literacy - 5th PV 2940 104.41 716.13 196.5576 1.46659 79.52127 
Valid N (listwise) 2940 
Notes: N = number of cases; PV = plausible value; Std. deviation = standard deviation. 


However, if weighted data are used with the IEAIDB Analyzer, the correct average of the ClL score 
in France is actually 498.71 (Figure 3.2). 


Figure 3.2: Example of weighted analysis using the IEA IDB Analyzer 


Average for PVCIL by IDCNTRY 


Country ID - Numeric Code N of Sum of Sum of Percent Percent PVCIL PVCIL Std.Dev 
cases TOTWGTS TOTWGTS (s.e.) (mean) (s.e.) 
(s.e.) 
France 2940 801969 12527.38 100,00 .00 498.71 2.35 80.45 
Table average : 7 : 100.00 .00 498.71 Pipa | 80.45 


Notes: N = number of cases; PVCIL = plausible value computer and information literacy; s.e. = standard error; Std Dev. = standard deviation. 
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3.3.1 Variance estimation variables in the ICILS 2018 international database 


Student-level, teacher-level and school-level variance estimation variables (or “jackknife variables”) 
are included in the ICILS 2018 international database (Tables 3.6, 3.7, and 3.8). 


Table 3.6: Student-level variance estimation variables in the student achievement and questionnaire file (BSG) 


Variable Description 

JKZONES Jackknife zone - student study 

JKREPS Jackknife replicate code - student study 
SRWGT1, Student jackknife replicate weight 1, 
SRWGT2,..., Student jackknife replicate weight 2, ... 
SRWGT75 Student jackknife replicate weight 75 


Note: For a full description of the variance estimation variables, see section 2.4.8 


Table 3.7: Teacher-level variance estimation variables in the teacher questionnaire file (BTG) 


Variable Description 

JKZONET Jackknife zone - teacher study 

JKRPET Jackknife replicate code - teacher study 
TRWGT1., ... Teacher jackknife replicate weight 1, ... 
TRWGT75 Teacher jackknife replicate weight 75 


Note: For a full description of the variance estimation variables, see section 2.4.8 


Table 3.8: School-level variance estimation variables in the school and ICT coordinator questionnaire file 
(BCG) 


Variable Description 

JKZONEC Jackknife zone - school study 
JKREPC Jackknife replicate code - school study 
CRWGT‘41,... School jackknife replicate weight 1, ... 
CRWGT75 School jackknife replicate weight 75 


Note: For a full description of the variance estimation variables, see section 2.4.8 


3.3.2 Selecting the appropriate variance estimation variables 

Different variance estimation variables must be applied depending on the type of data: 
e For all student level analyses, JKZONES and JKREPS should be used; 

e For all teacher level analyses, JKZONET and JKREPT should be used; and 

e For all school level analyses, JKZONEC and JKREPC should be used. 


Even for the same school, the variables at different levels of analysis can differ from each other and 
thus are not interchangeable. As is the case with weights, researchers should ensure to choose the 
correct jackknife variables when working with aggregated datasets. The level of analysis (student, 
teacher, or school) determines which variable to choose. 


When calculations are performed with the IEAIDB Analyzer, the correct variables will be selected 
automatically. However, users may want to use specialized software for those types of analysis that 
go beyond the range of the IDB Analyzer’s capabilities. In this case, the jackknife variables must 
be specified according to the requirements of the software. Usually, “zone” variables have to be 
specified as “stratum” or “strata” variables, while the “replicate” variables are commonly referred 
to as “cluster” variables. Frequently, software accepts direct use of the replicate weights. In such 
cases, the JKZONE and JKREP variables can be ignored. We strongly recommend that data users 
employ the replicate weights provided for all single-level analysis of ICILS 2018 data. 
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3.3.3 Example for variance estimation 

ot using the jackknife variables in data analysis will lead to standard errors that do not reflect 
correctly the uncertainty of point estimates. The following example illustrates the importance of 
using the JRR technique for research and analysis with ICILS 2018 data. 


Aresearcher may be interested in the average teacher age (variable T_AGE) in Chile. Using SPSS, 
they find that the (weighted) average teacher age is about 40 years and the standard error seems 
to be close to 0.05 years (see Figure 3.3). 


But using the JRR technique with the IEAIDB Analyzer, the correct estimate for the standard error 
is found to be more than nine times larger than indicated by the SPSS analysis (see Figure 3.4). 


Figure 3.3: Example of incorrect variance estimation in SPSS 


N Mean Std. error 
Teacher age 49720 39.65 
Valid N (listwise) 49720 


Notes: N = number of cases; Std. error = standard error. 


Figure 3.4: Example of weighted analysis using the [EA IDB Analyzer 


N of Sum of Sum of Percent Percent T_AGE T_AGE 
cases TOTWGTT TOTWGTT (s.e.) (Mean) (s.e.) 
(s.e.) 


1683 49837.87 1151.36 100.00 .00 40.02 


Notes: N = number of cases, TAGE = teacher’s age, s.e. = standard error. 


The standard methods of the SPSS base version can neither handle weights correctly for sampling 
variance estimation, nor can it take the clustered data structure into account. This means that not 
only standard errors but also all analyses that contain significance tests will be incorrect unless 
specialized software is used. 


3.3.4 Estimating sampling variance with jackknife repeated replication 


When population parameter pis estimated, then /, is its estimate, assuming all weighted sampled 
measurements have been used (i.e., applying TOTWGTS for the student population or TOTWGTT 
for the teacher population). Because all samples in ICILS 2018 are probabilistic, Uitself is arandom 
variable, and p is therefore estimated with a certain degree of precision. To account for this, we 
use JRR methodology to estimate the sampling variance of 


where 75 refers to the number of jackknife zones, and py; is the estimate of ~ using the i set of 
jackknife replicate weights. The standard error of Lis given by: 
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estimate the variance of the construct is given by: 
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3.3.5 Comparing groups and statistical significance testing 


Analyzing data by subgroups is common practice in research. However, if the aim is to review 


statisti 
insect 
subgroups ar 
or if al 


should 
schoo 
whet 


s.7 In 


In thi 
Beca 
deviat 


S 


secom 
ions Is 


Testin 
(1) 


Estimati 


ion 3.2.8. 


standard error esti 


er the surve 


section of t 


eunli 


ma 


deve 


opi 
yan 


parisono 


Esti 
stan 


(2) mati 


(3) Compar 


Absolute t-va 
(p < 0.05). In 
95 percent, p 


Note, however, that t-values are no proof of 


subgroups (a 


or not there is a difference is less than 95 percent. 


be compared 


ues larger th 
other words, 
redict that th 


mistake com 


cal differences among subgroups, users will need to proceed cau 
the sampling design has a direct impact o 
Even in the case of larger ef 
ely if the number of sa 
members of a subgroup belong to o 
te itself is not accurate 
have no fewer than 50 individuals 
ng research questi 
d sampling design support the respective research goals. 


ng the difference between two groups by sim 
estimates from each other; 


ng the standard error of the di 
dard error (the result of this divisio 


ing the t-value to the t-dis 


an 1.96 point 
if the absolu 
e difference i 


monly made 


The second step above (computing thes 
The method used to comp 


ute this stand 


.We can disti 


tiously. This is because 
e standard error of any estimate, as we pointed out 
fect sizes, statistically significant differences among 
mpled students or teachers within grouping cells is small 
nly avery small number of schools. Furthermore, the 
in these cases. As arule of thumb, an analysis group 
students or teachers) coming from at least 25 different 
ons and designs, we recommend that users evaluate 


nth 


his chapter, we consider comparisons of means, percentages, and percentiles. 
f other estimators such 
not as straightforward, this is not covered in this guide. 


as correlation or regression coefficients or standard 


g for significant differences between group estimates involves the following steps: 


ply subtracting the two group 


fference and the 
is called the “t-val 


n dividing the difference by its 
ue’): and 


n 


tribution. 


to significant differences onthe 95-percent certainty level 
te t-value is larger than 1.96, we can, with a probability of 
snot only present inthe sample but also in the population. 
the absence of a difference between two compared 
in statistical analysis); instead, the probability of whether 


tandard error of the difference) deserves special attention. 
ard error will depend on the composition of the groups to 


nguish between three cases. 


12 The JRR method measures sampling variance by comparing the variation between paired schools, which makes it 
important to have enough schools contributing to the computations. 
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Differences between independent samples 
Independent samples consist of sample subgroups that were not part of the same sampling frame. 
This axiom holds for comparisons across countries or among different explicit strata. 


The standard error of the difference SEgi¢ gy and the t-value for two independent groups a and b 
is computed by the IDB Analyzer as: 


SE if ab =y SEZ + SE¥, and 


__(a-b) 
SE gif ab 


t 


Differences between dependent samples 
Dependent samples consist of sample subgroups that were part of the same sampling frame. One 
example is gender groups. Assume that female and male students are sampled as part of the same 
explicit strata. For example, they attend the same school type (a feature that is relevant if used for 
explicit stratification), or they share the same teacher and school environment because they attend 
the same school. The sampling covariance between these subgroups will need to be considered 
during estimation of the standard errors. 


Using jackknife replication to estimate the standard error of the difference involves the following 
formula: 


[75 
SE aif ab -| 2 ((a'-b')-(a -b)?| 


| i=1 


Here, a and b represent the weighted averages (or percentages) in each of the two subgroups 
for the fully weighted sample, and a and b' are the weighted averages for the replicate samples. 


Where, with respect to ICILS 2018, there are differences in ClL scores, the measurement error 
also needs to be taken into account using the following formula: 


P 75 i 5 
SE dif ab = dot 2 j= ((,-b}) - (a,-b,) 


1 P 


Here, a, and b, represent the weighted subgroup averages in groups a and b for each of the P 
plausible values (P = 5), a} and bt are the subgroup averages within replicate samples for each of 
the P plausible values, and a, and b, are the means of the two weighted subgroup averages across 
the P plausible values. 


Obviously, manually computing the standard error estimates of these differences would be tedious. 
Asimpler solution is to model group differences with a regression, an approach which also builds 
in the covariance term. The IEA |DB Analyzer makes it easy to implement this approach for both 
variable types; t-values of group differences are part of the output. Section 4.4.2 of this guide gives 
a detailed explanation of the implementation of this method. 


Estimating standard errors of dependent samples by using the method for independent samples 
risks overestimating the standard error, thereby detecting fewer significant differences than are 
actually present. 
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Differences between group and combined-group estimates 

Researchers sometimes want to compare a group estimate with a combined estimate where the 
group of interest also contributes to the combined estimate (of independent groups). A typical 
example is that of comparing national average scores with the “country average” (an estimate 
based on data from all participating countries).!° In this case, the samples to be compared are not 
independent because the national mean contributes to the estimation of the international mean. 
The (adjusted) standard error estimate of this difference SE gi¢ i. can be computed as 


y ((N-1)2-1)-SE2+ 51, SE? 
N 


SE Gific = 


where SE, is the standard error for country c and SE, is the standard error for the k‘” of the N 
participating countries (or groups contributing to the combined estimate). 


Again, because the IEA IDB Analyzer does not offer this operation, it needs to be performed 
manually. 


3.3.6 Importance of using the correct variance estimation method 


The data structure must be taken into account when performing analyses, otherwise the analyses 
are likely to produce incorrect standard errors. Standard errors will be considerably underestimated 
inmost cases, and group differences will become significant even though they are not. The following 
example illustrates the importance of using the JRR technique when analyzing ICILS data. 


The ICILS 2018 international report presented country average CIL achievement scores for both 
females and males (Fraillon et al. 2020a, figure 3.7, p. 70). The difference between average scores 
for boys in Germany (511) and boys in Finland (516) was five ClL score points. The standard error 
has to be computed to verify whether the difference is statistically significant. Because the samples 
compared are independent, the following formula applies: 


= 2 7] 
SE aig ~ 4 SEGerman boys + SE Finland boys 


Inserting the standard errors of the ClL scores of both countries ( SE German boys = 3-6; SE¢iniand boys = 
3.6) into the formula, results in a standard error of the difference of 5.1. The next step is to divide 
the difference by its standard error to compute the t-value; here t = 0.98. Accordingly, the CIL 
average score difference between boys in Germany and boys in Finland might just be due to chance. 


However, estimating the standard errors of the same CIL scores on the assumption of simple 
random sampling (by, e.g., using SPSS) reveals that the standard errors are largely underestimated. 
When the total weights in the analysis are applied, the standard error estimations are 7 to 24 times 
smaller than the correct estimates (SEsps German boys = 0.15; SEsps Finland boys = 0-9). If these incorrect 
standard errors are used for hypothesis testing, the country difference becomes significant (the 
t-value would be 9.6). Failing to apply the weights while still treating samples as simple random 
samples would lead to underestimation of the standard errors (2.0 for Germany and 2.4 for Finland), 
and the difference would appear insignificant (the t-value would be 1.6). 


The effect of underestimating standard errors generally holds for all variables or types of analysis. 


13 Please see information about comparing North Rhine-Westphalia to the ICILS 2018 average in chapter 13 of the 
ICILS 2018 technical report (Fraillon et al. 2020b). 
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Figure 4.1: IEA IDB Analyzer main window 
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Table 4.1: Possible merging of data between different file types in ICILS 2018 
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Figure 4.2: IEA IDB Analyzer Merge Module: selecting countries 
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(4) To change how the country names appear in the outputs from the IDB Analyzer when 
analyzing data, users can select the Edit Country List button. Three columns are displayed: 
Code (numeric codes of the countries), ISO (the three-letter character codes for the 
countries), and Country Name. The values in the first two columns cannot be changed. Scrol 
down to the country that needs to be edited, click in the cell in the Country Name column 
and change the entry. Click OK to confirm the change (see Figure 4.3). To restore the default 
values, click the Restore Defaults button. Click OK to close the Edit County List window. 
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Figure 4.3: IEA IDB Analyzer Merge Module: editing country list 
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(5) Click the Next > button to proceed to the next step. The software will open the Select File 


Types and Variables tab o 
variables to be included in 


f the merge module (Figure 4.4) to select the file types and the 
the merged data file. 


(6) Select the file types for merging by checking the appropriate boxes to the left of the window. 


In the example, only the in 


ternational student questionnaire file is selected (see Figure 4.4). 


(7) Select the variables of interest from the Available Variables list in the left panel. Variables 
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tab-right arrow (pl) button selects all variables. Note that there are two tabs: Background 
Variables and Scores and ID and Sampling Variables. All CILand CT scores and all identification 
and sampling variables are selected automatically by the IEA | 
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the SPSS or SAS code used to perform the merge. In the example (Figure 4.4), the SPSS data 
fi tax file BSGALLI2.SPS are stored in the “C:\ICILS2018\Data\ 
Work” folder. When using SAS, the resulting files would be BSGALLI2.sas7bdat (data) and 
erged data file will contain student questionnaire data with 
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Figure 4.4: IEA IDB Analyzer Merge Module: selecting file types and variables 
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IEA IDB Analyzer: Merge Module - (Version 4.0.30) 


© Select File Types... € Select Variables... 


@GIEA 


ID and Sampling Variables (82) 


Available Variables: Selected Variables: 
School File : = - 
Tiger ||| ID and Sampling Variables (0) | | 
Teacher File | Background Variables and Scores (0) 


9 Output Files: C:\\CILS2018\Data\Work 
\BSGALLI3.* 


Background Variables and Scores (449) 


ABOUT YOU/When were you bor... | 
ABOUT YOU/When were you bor... 
ABOUT YOU/Are you a girl or a b... 
ABOUT YOU/Which of the followi... 
YOUR HOME AND YOUR FAMILY/... | 
YOUR HOME AND YOUR FAMILY/... | 
YOUR HOME AND YOUR FAMILY/... 
YOUR HOME AND YOUR FAMILY/... 
YOUR HOME AND YOUR FAMILY/... | 
YOUR HOME AND YOUR FAMILY/... 


YOUR HOME AND YOUR FAMILY/... | 


YOUR HOME AND YOUR FAMILY/... _ 
YOUR HOME AND YOUR FAMILY/... | 
YOUR HOME AND YOUR FAMILY/... | 
YOUR USE OF ICT/How long have... | 
YOUR USE OF ICT/How long have... 
YOUR USE OF ICT/How long have... 


YOUR USE OF DIGITAL DEVICES/... 
YOUR USE OF DIGITAL DEVICES/... | 
YOUR USE OF DIGITAL DEVICES/... 
YOUR USE OF DIGITAL DEVICES/... | 
YOUR USE OF DIGITAL DEVICES/... 


Return to Main Menu Help 
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from the Run menu will execute the syntax. The | 
is about to overwrite an existing file in the specified folder. 


EA 


DB Analyzer stores the SPSS syntax file in the specified folder and opens it ina SPSS Syntax 
Editor window (Figure 4.5) ready for execution. The syntax file can be executed by openi 
the Run menu of SPSS and selecting the All menu option. In SAS, hitting the Submit opti 


EA IDB Analyzer will display a warning i 
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Figure 4.5: SPSS Syntax Editor with merge syntax produced by the IEA IDB Analyzer Merge Module 
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@ ICILS2018_Student.sps - IBM SPSS Statistics Syntax Editor 
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Active DataSet 
include file = "C-\Users\ekaterina.mikheeva\AppData\Roaming\IEA\IDB Analyzer\IDBAnalyzerCountries ieasps” 


indir="CICILS2018\Data"/ 


outdir="C:\ICILS2018\Data\Work"/ 
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include file = "C-\Users\ekaterina.mikheeva\AppData\Roaming\IEA\IDBAnalyzerV4\bin\Data\Templates\SPSS_Macros\IDBAnalyzer.ieasps”. 


infile=BSGCHLI2 BSGDEUI2 BSGDNKI2 BSGDNWI2 BSGFINI2 BSGFRAI2 BSGITAI2 BSGKAZI2 BSGKORI2 BSGLUXI2 BSGPRTI2 BSGRMOI2 BSGURY!: 


IBM SPSS Statistics Processoris ready | |  |Unicode:ON |In1Col0| | 


Once SPSS or SAS has completed its execution, it is important to check the SPSS output window 
or SAS log for possible warnings. If warnings appear, they should be examined carefully because 
they might indicate that the merge process was not performed properly and that the resulting 
merged data file might not be as expected. 


4.2.2 Merging school and student data files 


The ICILS 2018 school samples were designed to optimize the student samples and the student- 
level estimates. Therefore, it is preferable to analyze school variables as attributes of students, 
rather than as elements in their own right. However, the school samples are representative 
probability samples of schools within each participating country and so it is possible to compute 
weighted numbers of schools with particular characteristics for providing reasonable estimates 
of percentages and means across the populations of schools in each country. 


When merging school and student questionnaire data files, the school data will be disaggregated 
to student level, so that the school data become an attribute of the students in that school. 


To merge the school and student questionnaire data filles, select both the school file and student file 
types. The variables of interest to be included in the merged data file need to be selected separately 
by file type using the same set of instructions as described in section 4.2.1. The ID and sampling 
variables will be selected automatically. Please note that when merging student and school data, 
only the total student weight (TOTWGTS) variable will be included in the merged flle, but not the 
total school weight (TOTWGTC). An analysis using school variables on student level together with 
student-level variables will use the total student weight and will not allow the researcher to make 
inferences for the school themselves. The interpretation of the results will be about students 
who study in schools with certain characteristics. For example, if merged student and school data 
uses the principals’ gender as the grouping variable, the total student weight would be selected as 
weighting variable. The results then would be interpreted as percentages of students who study in 
schools where the school principal is male or female: e.g., In Chile, 47% of grade 8 students study 
in schools with male principals, and 52% study in schools with female principals (the reason the 
numbers do not add up to exactly 100% is because of rounding). 


46 


ICILS 2018 USER GUIDE FOR THE INTERNATIONAL DATABASE 


4.2.3 Merging school and teacher data files 


Merging the school and teacher data files follows the same procedure as merging the school and 


=! cy 


eed to be 


sampl 


student data files. School data will be disaggregated to the teacher level by adding the respective 
chool level variables to each teacher record. To merge teacher questionnaire and school 
uestionnaire data files, perform steps 1 to 4 as described in section 4.2.1. Then, simply select both 
e types in the second window of the IEA IDB Analyzer Merge Module. The variables of interest 


selected separately for both file types, as follows: 


(1) Check the checkbox next to the teacher file type so that it appears checked. The ID and 


ing variables are selected automatically and already listed in the right panel. 


(2) elect the variables of interest from the left panel and click the right arrow (®) button to move 


these 


(3) ext, 


button to select all available variables. 


variables to the Selected Variables panel on the right. Click the tab-right arrow (pl) 


check the checkbox next to the school file type. Based on the country selection, the 


EA IDB Analyzer might display a warning that certain countries do not have data for the 
selected regional questionnaire. Close the warning message and select the variables of 
interest from the Background Variables and Scores panel in the same manner as described in 


step 1 and 2. 


(4) Define/Modify the desired name of the merged data file and the folder where it will be 
stored in the Output Files field. The IEA IDB Analyzer will create an SPSS syntax file (“SPS) 


of the 


same name and in the same folder with the code necessary to perform the merge. 


(5) Click on the Start SPSS button to create the SPSS syntax file that will produce the required 
merged data file, which can then be run by opening the Run menu of SPSS and selecting the 


All op 


tion. 


4.2.4 Merged data files for the example analyses 


Tocarry ou 


t the sample analyses described in this chapter, users should expect to find the following 


merged data flles, including all available context and background variables and scores: 
(1) BSGALLI2.SAV Merged student (BSG) data files for all countries; 

(2) BTGALLI2.SAV Merged teacher (BTG) data files for all countries; and 
(3) BSG BCGALLI2.SAV Merged school (BCG) and student (BSG) data files for all 


countries. 
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4.3. Performing analyses with the IEA IDB Analyzer 


The IEA | 


or on any original | 
odule of 


Analysis 


EAs 
the | 


EA IDB Analyzer (Table 4.2 


i 


DB Analyzer can perform statistical analyses on any files created using the Merge Module 
tudy data file. There are several statistical procedures available in the 


Table 4.2: Statistical procedures available in the Analysis Module of the IEA [DB Analyzer 


Statistic type 


Description 


Percentages and Means 


Computes percentages, means, and standard deviations for selected 
analysis variables by subgroups defined by grouping variable(s). 
Plausible values can be included as analysis variables. 


Percentages only 


Computes percentages by subgrou 


ps defined by grouping variable(s). 


Linear regression 


Co 
var 
grouping variabl 
ori 


ndependent varia 


mputes linear regression coeff 
iables to predict a dependent variab 
Plausible valu 


e(s). 


bles. 


cients for selected independent 
e by subgroups defined by 
es can be included as dependent 


Logistic regression 


* 


Co 
var 
grouping variabl 
or independent 
plausible values 
suc 


has one of the IC 


mputes logistic regression coefficients for se 
iables to predict a 


e(s). 
vari 
will 


be di 


dependent v. 
Plausible valu 
ables. When 


LS 2018 international benc 


ected independen 
e by subgroups defined by 
es can be included as dependent 
used as a dependent variable, 
zed using a specified cut-point, 
hmarks. 


ariab 


chotomi 


Benchmarks 


Com 


variable(s). 


putes percentages of students meeting a set of user-s 
Achievement proficiency levels by subgroups de 


ed 
Ng 


peci 


fined by groupi 


Correlations 


Co 
for 
Pla 


7 
selected va 


putes means, standard deviations, and 
ables by subgro 
usible values can be included as analysis varia 


fcien 
iable 


correlation coet 
by grouping var 
bles. 


ups defined 


Percentiles 


Computes the 
the distributio 
variable(s). Plau 


score points th 


at separate a given proportion 


of scores, by s 
sible values can 


ubgroups defined by the groupi 
be included as analysis vari 


ng 
ables. 


Group 


Differences by Performance* 


Compu 
conditional on 


tes differences in ana 
the Achievemen 


nalysis variable, between groups, 


or performance levels. 


Note: * The statistics types Logistic Regression and Group Differences by Performance will not be covered in this chapter. 


Refer to tl 


The IEA | 
statistica 
use of san 
(JRR) method. 
speci 


accurate estima 


e IEAIDB Analyzer Help manual for informa 


DB Analyzer has a Help Manual th 
procedures offered in the analysis modu 
npling weights and standard errors are com 


hed with o 
are performed fi 


ion abou 


at prov 


Percentages, means, linear regres 


ve times (once for each plausible va 
tes of CILor CT with standard errors 


ides exte 
le of the 
puted us 


ue) andt 


sions, co 
r without achievement scores. When CIL or 


these types of analysis. 


sive descriptions of all features. All 
EA IDB Analyzer make appropriate 
ing the jackknife repeated replication 
rrelations, and percentiles may be 
CT scores are selected the analyses 
he results are aggregated to produce 


errors. To conduct analyses using achievement scores, select t 


Value Option d 
selected froms 


rop-down menu. The diverse variab 


that incorporate both sampling and imputation 


he Use PVs option from the Plausible 


es required to perform an analysis need to be 
pecific variable fields according to their purpose (see Table 4.3). 
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yzer 


Field 


Description 


Grouping Variables 


This is alist of variables to define subgroups of interest. The list must consist of 
at least one grouping variable. By default, the IEAIDB Analyzer includes the 
variable IDCNTRY used to distinguish the participating countries. Additional 
variables may be selected from the available list. If the Exclude Missing from 


Analysis option is checked, only cases that have non-missing values in the 


grouping variables will 


values become r 


eporting categories. 


be used in the analysis. If it is not checked, missing 


Analysis Variables 


This is a list of variables for which means, percentages, correlations, or 


percenti 


can be selected 


les are to be computed. Usually, more than one analysis variable 
. To compute statistics based on achievement scores, it i 


necessary to select the 
down menu, and select the Achievement Scores of interest inthe Plausible 


Values fi 


eld. 


S 
Use PVs option in the Plausible Value option drop- 


Plausible Values 


This section is used to i 


Achievement Scores are the analysis variable for computing statistics. Selec 


the Use 


specifying the Achievement Scores in the Plausible Values field. 


PVs option int 


dentify the set of plausible values to be used when 


he Plausible Value option drop-down menu before 


Independent Variables 


This is a list of variables to be treated as independent variables for a linear 
or logistic regression analysis. More than one independent variable can be 
selected. Categorical variables and continuous variables can be specified as 
independent variables. When specifying categorical variables as independen 
variables, they can be 


reated either by “effect coding” or “dummy coding” 


using the Contrast drop-down menu. Achievement Scores also can be 

as an independent variable. To specify Achievement Scores as 
endent variable, it is necessary to select the Use PVs option in the 
Plausible Value option drop-down menu and select the CIL/ CT achievement 


included 
an indep 


SCOres O 


f interest in the Plausible Values field. 


Dependent Variable 


This is the variable to be used as the dependent variable when a linear or 
logistic regression ana 
be listed and can be either a context or background variable or CIL/CT 


achievement scores. To 


select th 
click ont 


and select th 


e Use PVs opti 
he Plausible Va 


ysis is specified. Only one dependent variable can 


use Achievement Scores as the dependent variable, 
on in the Plausible Value option drop-down menu, 
ues radio button in the Dependent Variable section, 


e Achievement Scores of interest in the Plausible Values field. 


Achievement Benchmarks 


These are the values tha 


selected 


specified, each separated by a blank space. A drop-down menu is available 
to select the ICILS 2018 international benchmarks. 


t will be used as cut points on the Achievement Scale, 


in the Plausible Values section, for computing the percentages 
of students meeting the specified benchmarks. Multiple cut points can be 


Percentiles 


These are the percentiles that will be calculated from the distribution of 
a continuous analysis variable selected in the Analysis Variables section. 
Achievement Scores can also be selected as an analysis variable. Select the 
Use PVs option in the Plausible Value option drop-down menu and select 
the Achievement Scores of interest in the Plausible Values field. Multiple 


percenti 


les can be spec 


ified, each separated by a blank space. 


Weight Variable 


This is the sampling weight variable that will be used in the analysis. The 
EA IDB Analyzer automatically selects the appropriate weight variable for 
analysis based on the file types included in the merged data file. Generally, 
this willbe TOTWGTS for analysis on student level. When analyzing teacher 
data TOTWGTT will be used. Chapter 3 of this user guide provides more 
information on the ICILS 2018 sampling weights. 
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4.4 Performing analyses with student-level variables 


Many analyses of the |CILS 2018 data can be undertaken using student-level data only. This section 
presents examples of analyses used to produce tables for the ICILS 2018 international report 
(Fraillon et al. 2020a); these include examples of computing percentages only, and percentages and 
means, undertaking linear regression analyses, and computing percentages of students reaching 
proficiency levels. We also describe how to conduct correlation analyses. 


4.4.1 Student-level analysis without achievement scores 


As an example of how to undertake a student-level analysis without achievement scores, we here 
replicate the analysis of the students’ reported age at the time of testing that was presented in the 
ICILS 2018 international report (Table 4.4; see Fraillon et al. 2020a, table 3.4, p. 75). This example 
focuses on the average age at the time of testing (with appropriate standard errors) and, therefore, 


we compute means without achievement scores. 


Table 4.4 : Example of student-level analysis without CIL achievement scores originally p 


2018 international report 


blished in the ICILS 
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Country Average age ClL achievement distribution Average CIL score ICT development index (IDI) 

of students score (and country rank) 

iniyears 190 200300 aS _ 600, 700 

Denmark’ * 49 | 553 (20) 871 (4) 
Korea, Republic of 42 — 542 (3.1) A 8.85 (2) 
Finland 48 is Ba 531 (3.0) A 7.88 (22) 
Germany 45 | 518 (2.9) «A 8.39 (12) 
Portugal't * 41 ca 516 (26) 7.13 (44) 
France 38 — 499 (2.3) 8.24 (15) 
Luxembourg 45 = : 482 (0.8) Vv 847 (9) 
Chile 41 — a 476 (3.7) 6.57 (56) 
Uruguay 43 ———— 450 (43) W 7.16 (42) 
Kazakhstan? 43 | 395 (54) 6.79 (52) 
ICILS 2018 average 44 Below L1 ils L3 L4 496 (1.0) 
Testing at the beginning of the school year 
Italy 3.3 = 461 (28) W 7.04 (47) 
Not meeting sample participation requirements 
United States 42 aT 519 (1.9) 8.18 (16) 
Benchmarking participants meeting sample participation requirements 
Moscow (Russian Federation)? 48 | —— i —| 549 (2.2) A 7.07 (45)? 
North Rhine-Westphalia (Germany)? 44 Sa 515 (26) 8.39 (12) 


Notes: ICT development index (IDI) score and country rank data relate to 2017 (source: ITU 2019). 

Standard errors appear in parentheses. Dotted vertical lines indicate the cut-points between proficiency levels. 
Met guidelines for sampling participation rates only after replacement schools were included. 

' Nearly met guidelines for sampling participation rates after replacement schools were included. 
National defined population covers 90% to 95% of the national target population. 

? Data relate to all of Russian Federation. 

’ Data relate to all of Germany. 


Source: Fraillon et al. (2020a, table 3.4, p. 75). 


5th 


25th 


Percentiles of performance —] 


75th 95th 


Mean and confidence interval (+2SE) 


Replicating the results in this table involves several steps. The student questionnaire data codebook 
reveals that the student questionnaire data derived variable S AGE is the numeric variable 


reporting the age of students at the time of testing. 


A Achievement significantly higher 
than ICILS 2018 average 

VY Achievement significantly lower 
than ICILS 2018 average 
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After creating the merged data file for the analysis, the Analysis Module of the IEA IDB Analyzer 
enables the user to perform the analysis using the following steps (see Figure 4.6): 


7) The variable IDCNTRY is selec 


(1) Open the Analysis Module of the IEA IDB Analyzer. 

(2) Select the merged data fille BSGALLI2.SAV as the Analysis File by clicking the Select button. 
(3) Select ICILS (Using Student Weights) as the Analysis Type. 

(4) Select Percentages and Means as the Statistic Type. 
( 
( 
( 


5) Select None Used as the Plausible Value Option. 


6) Set 1as the Number of Decimals (as in the report). 


ted automatically as Grouping Variables. No additional 


grouping variables are needed for this analysis. Leave the default option Exclude Missing 


From Analysis selected. 
(8) Click the Analysis Variables fie 


Id to activate it. Select S AGE from the list of available 


variables and move it to the Analysis Variables field by clicking the right arrow (>) button in 


this section. 


© 
4 


although the IEA IDB Analyzer in 


he Weight Variable is automatically defined by the software. As this example analysis uses 
student questionnaire data, TOTWGTS is selected by default. The 75 replicate weights will 
also be involved in the analysis for computing the correct estimates of the standard errors, 


terface does not indicate or list them individually. 


(10) Specify the name and the folder of the output files in the Output Files field by clicking the 


Define/Modify button. The IEA 


(iv) an Excel file with these same r 
n SPSS data file, and (vi) an Exce 


w 


IDB Analyzer will use this name and folder to create six 


output files. (i) An SPSS syntax file that contains the code for performing the analysis, and, 
after running the syntax flle, (ii) an SPSS data fille with the results, (iii) an SPSS output file, and 


esults. Another two filles contain significant test results: (v) 
| file. 


(11) Press the Start SPSS button to create the SPSS syntax file. The file will open in an SPSS syntax 
window. The syntax file will be executed by opening the Run menu of SPSS and selecting the 


All option. If necessary, the IEA | 
already existing files. 


Note that IEA IDB Analyzer also disp 


DB Analyzer will prompt the user to confirm overwriting 


ays an average statistics for all countries included in the 


analysis at the end of the results tables (“Table Average’). 
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Figure 4.6: IEA IDB Analyzer setup for example student-level analysis without plausible values 


HEE IEA 1DB Analyzer: Analysis Module - (Version 4.0.30) 


@ Analysis File: C:\ICILS2018\Data\Work\BSGALLI3.sav Select 
12) 
Analysis Type: Statistic Type: Plausible Value Option: Number of Decimals: Show Graphs 
[ICILS (Using Student Weights) » | [Percentages and Means ~ | |None Used +] [2 ~| |Yes : 


GIEA 


© Select Variables: 


Country ID - Alpha Code 
ABOUT YOU/When were you born/Month 
ABOUT YOU/When were you born/Year 
ABOUT YOU/Are you a girl or @ boy 
ABOUT YOU/Which of the following [levels of ed... 
YOUR HOME AND YOUR FAMILY/In what countr... 
YOUR HOME AND YOUR FAMILY/In what countr... 
YOUR HOME AND YOUR FAMILY/In what countr... 
YOUR HOME AND YOUR FAMILY/What language... 
YOUR HOME AND YOUR FAMILY/What is the hig... 
YOUR HOME AND YOUR FAMILY/What is the hig... 
YOUR HOME AND YOUR FAMILY/About how ma... 
YOUR HOME AND YOUR FAMILY/How many of t... 
YOUR HOME AND YOUR FAMILY/How many of t... 
YOUR HOME AND YOUR FAMILY/Internet conne... 
YOUR USE OF ICT/How long have you been usin... 
YOUR USE OF ICT/How long have you been usin... 
YOUR USE OF ICT/How long have you been usin... 
YOUR USE OF DIGITAL DEVICES/Who mainly tau... 
| Bis2c178 YOUR USE OF DIGITAL DEVICES/Who mainly tau... 


© Output Files: | C:\\CILS2018\Data\analysis\S_AGE* Modify 


) Grouping Variables: [¥] Exclude Missing From Analysis 


> Gf IDCNTRY Country ID - Numeric Code 


) Separate Tables by: 


ccs 


@) Analysis Variables: 


KBS_AGE Age of student 3 


|» B® totwars Final Student Weight 4 


Return to Main Menu Help 


| Start spss 


SPSS _ICILS (Using Student Weights)  Percentagesand Means None Used _ekaterina.mikheeva 


In this example, each country’s average for the S AG 
The IDB Analyzer outputs the number of valid cas 


E variable is reported for all sampled students. 
es, sum of weights of the sampled students, 


and their percentage, mean, and standard deviation, as well as the respective jackknife standard 


errors (Figure 4.7). Inthis case, Chile has valid data for 3092 students and these sampled students 
represent a population of 225,453 students. Chilean students were, on average, 14.4 years old at 
the time they took the ICILS 2018 assessment, with a standard error of less than 0.01. Less than 
0.01% of Chilean students did not report their age at the time of testing. 
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4.4.2 Student-level analysis with achievement scores 


Inour second example, we replicate another set of results presented inthe ICILS 2018 international 
report (Table 4.5; see Fraillon et al 2020a, table 3.7, p. 80). In this case, the example investigates 
the relationship between students’ gender and CIL, the latter being represented by a set of five 
plausible values. Since the results are based on plausible values, these must be included when a 
user creates the file using the merge module, to indicate that the analysis will make use of CIL 


achievement scores when the user specifies the analysis type. 


Table 4.5: Example of student-level analysis with CIL achievement scores originally published in the ICILS 


2018 international report 
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Country Mean scale score Mean scale score Difference Gender difference 
females males (females - males) 0 25 50 
Chile 480 (4.1) 472 (4.7) 8 (48) 
Denmarkt * 561 (2.2) 545 (2.8) 16 (3.0) [Eas 
Finland 545 (3.2) 516 (3.6) 29 (3.6) ae cee 
France 511 (2.9) 487 (28) 24 (33) ——— ss pe 
Germany 526 (3.1) 511 (3.6) 16 (3.3) —aay higher 
azakhstan* 399 (5.6) 391 (5.9) 8 (4.1) Sa 
orea, Republic of 563 (3.4) 524 (3.9) 39 (4.3) a eS 
Luxembourg 494 (1.5) 471 (1.3) 23° (2.3) ESSSS_S4 
Portugal't* 522 (2.6) 511 (3.2) 11 (3.0) —— 
Uruguay 453 (4.9) 448 (4.9) 5 (4.5) 
ICILS 2018 average SOsm (iit) 488 (1.2) 18 (1.2) _———) 
Testing at the beginning of the school year 
Italy 469 (3.6) 454 (3.1) 16 (3.7) <n 
Not meeting sample participation requirements 
United States 531 (2.0) 508 (2.3) 23 (2.1) =<—— 
Benchmarking participants meeting sample participation requirements 
Moscow (Russian Federation) 552 (2.5) 546 (2.7) 6 (2.7) = 
North Rhine-Westphalia (Germany) 517 (3.3) 513 (3.2) 4 (3.8) 


Notes: Standard errors appear in parentheses. Because results are rounded to the nearest whole number, some totals may appear 
inconsistent. Statistically significant differences (p < 0.05) between subgroups are shown in bold. 
Met guidelines for sampling participation rates only after replacement schools were included. 
+ Nearly met guidelines for sampling participation rates after replacement schools were included. 
‘National defined population covers 90% to 95% of the national target population. 


Source: Fraillon et al (2020a, table 3.7, p. 80). 


QO 


ategorical information on the gender of the student. 


IDB Analyzer enables the user to replicate the analysis of gender differences 


[Bl Gender difference statistically significant at p < 0.05 level 


The codebooks reveal that the variable S SEX in the student questionnaire data files contains 


The Percentages and Means analysis type with the Use PVs option activated computes percentages 
and mean CIL scores based on plausible values and their respective standard errors. The IEA 


in ClL scores. After 


opening the analysis module and selecting the BSGALLI2.SAV data file, the steps in the IEA IDB 


Analyzer are as follows (Figure 4.8 shows how the analysis module looks when the settings for 


this example analysis are entered correctly): 


1) Open the Analysis Module of the IEA IDB Analyzer. 


3) Select ICILS (Using Student Weights) as the Analysis Type. 


records with missing grouping variables from the analysis. 
(5) Select Use PVs as the Plausible Value Option. 


(6) Add the variable S SEX as a second Grouping Variable. The variable | 
selected automatically as a grouping variable. 


2) Select the merged data file BSGALLI2.SAV as the Analysis File by clicking the Select button. 


4) Select Percentages and Means as the Statistic Type. By default, the program will exclude 


DCNTRY is always 


Gender difference not statistically significant 
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(7) Specify the achievement scale to be used (in this example CIL) for the analysis by clicking the 
Plausible Values field to activate it. Select PVCILO1-05 from the list of available variables 
and move it to the Plausible Values field by clicking the right arrow (») button in this section. 


(8) The Weight Variable is automatically selected by the software. As this example analysis 
uses student questionnaire data, TOTWGTS is included in the analysis by default. 


(9) Specify the name and the folder of the output files in the Output Files field by clicking the 
Define/Modify button. 

(10) Click the Start SPSS button to create the SPSS syntax file. The file will open in an SPSS syntax 
window. The syntax file will be executed by opening the Run menu of SPSS and selecting the 
All menu option. If necessary, the IEA IDB Analyzer will prompt the user to confirm or cancel 
overwriting already existing files. 


Figure 4.8: IEA IDB Analyzer setup for example student-level analysis with achievement scores 


EEE IEA IDB Analyzer: Analysis Module - (Version 4.0.30) = oO x 
@ Analysis File: | C:\ICILS2018\Data\Work\BSGALLI3.sav Select 
2] 
Analysis Type: Statistic Type: Plausible Value Option: Number of Decimals: Show Graphs Ee A 
JICILS (Using Student Weights) | | Percentages and Means ~ | | Use PVs aie ~| [es =] 


© Select Variables: 


©) Grouping Variables: [YJ Exclude Missing From Analysis 


Country ID - Alpha Code ] [51>] [PP IOCNTRY Country ID - Numeric Code 
ABOUT YOU/When were you born/Month J) Gs sx Secotsindent 

ABOUT YOU/When were you born/Year 
ABOUT YOU/Are you a girl or a boy 
ABOUT YOU/Which of the following [lev... 
YOUR HOME AND YOUR FAMILY/In wha... 
YOUR HOME AND YOUR FAMILY/In wha... 
YOUR HOME AND YOUR FAMILY/In wha... © Plausible Values: 
YOUR HOME AND YOUR FAMILY/What L... 


@® Separate Tables by: 


YOUR HOME AND YOUR FAMILY/What i... 
YOUR HOME AND YOUR FAMILY/What i... 


| | APVCILO1-05 1ST TO 5TH PV 4 


(A) Weight Variable: 


YOUR HOME AND YOUR FAMILY/About... 
YOUR HOME AND YOUR FAMILY/How m... 
YOUR HOME AND YOUR FAMILY/How m... 
YOUR HOME AND YOUR FAMILY/Interne... 

¥ YOUR USE OF ICT/How long have you b.. 
<B1S26168 YOUR USE OF ICT/How long have you b... 
1S2G16C YOUR USE OF ICT/How long have you b... 
3 IS2G17A YOUR USE OF DIGITAL DEVICES/Who m... 

} & 1S2G17B YOUR USE OF DIGITAL DEVICES/Who m... Fy 


| 


| Name Descriptio! ! 
> G@@ToTWeTs Final Student Weight q 


© Output Files: | C:\ICILS2078\Data\analysis\S_SEX* Modify Return to Main Menu Help 


QO |_ Start spss 


SPSS __ICILS (Using Student Weights) Percentagesand Means Use PVs __ ekaterina.mikheeva 


The IEA IDB Analyzer outputs each country’s results split by gender, i.e. the values of the S_ SEX 
variable (Figure 4.9). The countries are identified in the first column and the second column 
describes the category of S SEX being reported. The third column reports the number of valid cases 
and the fourth and fifth the sum of weights of the sampled students and the standard error of that 
sum. Two columns report the percentage of students in each category and the associated standard 
error, followed by the estimated mean CIL score and associated standard error. The standard 
deviation of the CIL achievement scores and standard error are reported in the last two columns. 


For example. In Denmark, 49.46% of the target population students are girls and 50.54% are boys; 
the mean ClL achievement score is 560.84 (standard error of 2.24) for girls and 544.62 (standard 
error of 2.81) for boys (Figure 4.9). 
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The IEA | 
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DB Analyzer also provides output files for all analysis types in 


Excel format. | 


[hese 


files are located in the folder specified in the Output Files fleld. For the Percentages and Means 
analysis type (and as well for the Benchmark analysis type) the program generates an Excel file 
with significance test results for differences of percentages as well as differences in means (for 
an example, see Figure 4.10). 


Figure 4.10: IEA IDB Analyzer Excel output including significance test results for example studen 
analysis with CIL achievement scores 


t-level 


4, oA Cae: E E G H | J K [L, M N fe} | B Q R s is 
1 IDCNTRY refgroup compgrcemnpy cmnpv mnpvdiff pct cpct pctdiff mnpv_secmnpv_se mnpvdiff_sepct_se cpct_se pctdiff_se pctdiff_t mmnpvdiff_igroupvar dvar weight 
2 Chile Boy Boy 471.88 471.88 0.00 50.97 50.97 0.00 4.66 4.66 #NULL! 1.66 1.66 #NULL! #NULL! #NULL! S SEX PVCIL TOTWC 
3 |Chile Boy Girl 471.88 480.14 8.26 50.97 49.03 -1.95 4.66 4.14 4.81 1.66 1.66 3.32 -0.59 1.72 S_SEX PVCIL TOTWC 
4 Chile Girl Boy 480.14 471.88 -8.26 49.03 50.97. 1.95 4.14 4.66 4.81 1.66 1.66 3.32 0.59 -1.72 S_SEX PVCIL TOTWC 
5 |Chile Girl Girl 480.14 480.14 0.00 49.03 49.03 0.00 4.14 4.14 #NULL! 1.66 1.66 #NULL! #NULL! #NULL! S_SEX PVCIL TOTWC 
6 Denmark Boy Boy 544.62 544.62 0.00 50.54 50.54 0.00 2.81 2.81 #NULL! 0.92 0.92 #NULL! #NULL! #NULL! S_SEX PVCIL TOTWC 
7 Denmark Boy Girl 544.62 560.84 16.2250.54 49.46 -1.08 2.81 2.24 3.03} 0.92; 0.92 1.83 -0.59 5.36 S_SEX PVCIL TOTWC 
8 Denmark Girl Boy 560.84 544.62 -16.22 49.46 50.54 1.08 2.24 2.81 3.03 0.92 0.92 1.83 0.59 -5.36 S SEX PVCIL TOTWC 
9 Denmark Girl Girl 560.84 560.84 0.00 49.46 49.46 0.00 2.24 2.24 #NULL! 0.92 0.92 #NULL! #NULL! #NULL! S_SEX PVCIL TOTWC 
10 Finland Boy Boy 516.34 516.34 0.00 50.62 50.62 0.00 3.65 3.65 #NULL! 0.81 0.81 #NULL! #NULL! #NULL! S SEX PVCIL TOTWC 
11 Finland Boy Girl 516.34 545.34 29.00 50.62 49.38 -1.24 3.65 S28 3.60 0.81 0.81 1.62 -0.77 8.05 S SEX PVCIL TOTWC 
12 Finland Girl Boy 545.34 516.34 -29.00 49.3850.62 1.24 324 3.65 3.60 0.81 0.81 1.62 0.77 -8.05 S_SEX PVCIL TOTWC 
13 Finland Girl Girl 545.34 545.34 0.00 49.38 49.38 0.00 3.21 3.21 #NULL! 0.81 0.81 #NULL! #NULL! #NULL! S SEX PVCIL TOTWC 

14 France Rav Rov __AR6 8? ARB R? _NNN 50225022 NON _2?85 __?85__#NUII1 _ Tm 064  #NULLI #NULLT _#NUNIT SS SFX PVCI_ TOTWC ~ 
@ ¢ > 


$_SEX PVCIL by S SEX Sig 


Notes: The output shows only a few countries, and is for illustrative purposes. refgroup = the label of the re 
group; compgroup = the label of the comparison group; mnpv and cmnpv = the mean plausible value of the r 


erence 
eference 


and the comparison group; pct and cpct = percent of people in the reference and the comparison group; mnpv_se and 


cmnpv_se = standard errors for mnpv and cmnpv; pct_se and cpct_se = standard errors for pct and cpct; mnpvdiff = 
erence group on the analysis variable; mnpvdiff_se = s 
of mnpvdiff; mnpvdiff_t = t-statistics for the mean difference between the reference and comparison group; 
difference between the percent of people in the comparison group and the reference group; pctddiff_se = s 
rence between the reference and comparison group; g| 
being compared; dvar = name of the dependent variab 


difference between the comparison group and the re 


of pctddiff; pctddiff = t-statistics for the percent diffe 
name of the variable that defines the groups that are 


The Excel file provides significance test results for all combin 
specified in the analysis module. In this example, this comprises al 
boys, boys with girls, girls with boys, and girls wi 
boys (or vice versa). In 

ready established fron 
544.62). In our example, row 6 of the 

16.22 score points lower than girls on 
provides the t-value, which in this case 
is statistically significant, as the absolut 


boys with 
girls with 
boys was a 


Column Q 
fference 


dit 


4.4.3 Student-level linear regression analysis 
The lEAID 


Denmark, the dit 
nthe SPSS outpu 
Excel file (Fi 


B Analyzer is able to calculate multiple li 


the ClL ach 
is-5.3 


fference | 
t(see 
gure 4 


ievem 


e valu 


th gi 
nClLac 
Figure 
1.10) shows 
ent sca 
6. Opti 
e of that t-val 


ations o 


combinati 


near regressions be 


e (wit 


that, 


n 


e. 


tandard error 
pctddiff = 
andard error 
roupvar = 


f grouping variables that were 
ons of students’ gender: 
rls. The interesting comparison is 
hievement between girls and 
9: girls score 560.84, boys score 
in Denmark, boys score 
h a standard error of 2.24). 
ng for a 95% confidence level, this 
ue (5.36) is =1.96. 


tween dependent variables 


and aset of independent variables. This section demonstrates an example for a regression analysis 


with achievement scores u 


BSGALLI2.SAV. 


The | 


SC 


EA IDB Analyzer can 
ores, but no example wil 


sing student-level variables selected in the example merged data file 


also be used to compute regression analysis without achievement 
be described here, as the steps are similar to those described for a 


regression analysis with achievement scores. The only difference between the two analyses is 


th 


at, if conducting the “with 


instead of Use PVs from the Plausible Value Option drop-down menu. 


out achievement scores” analysis, the user needs to select None Used 
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Linear regression with continuous independent variables 


(9) 


took the 


The exam 
as follows (Figure 4.11 shows how 
this example analysis are entered): 


Open the Analysis Module of 


ple regression analysis is performed using the analysi 
the analysis module should look when the proper settings for 


ariable S TLANG, which ind 


05), using the 
the merged data file BSGALLI2.SAV and 


the IEA IDB Analyzer. 


s module of th 


e| 


EA | 


rornott 
redictor 
weighting variable 
the standard errors 


D 


In our third example, we do not replicate any results presented in the ICILS 2018 international 
report; instead we examine the language spoken at home as a predictor of CT achievement. The 
linear regression analysis uses the v 
CILS 2018 assessment in the language spoken at h 
plausible values for CT achievement (PVCTO1 through PVCT 
TOTWGTS. The data are read from 
are computed based on 75 sets of replicate weights. 


he student 
of the five 


B Analyzer 


Specify the data file BSGALLI2.SAV as the Analysis File by clicking the Select button and 


selecting the file from the locat 
Select ICILS (Using Student Weights) as the Analysis Type. 


Select Linear Regression as th 


on it has been saved. 


e Statistic Type. 


Select Use PVs as the Plausible Value Option. 


The variable IDCNTRY is se 


gro 


uping variables are needed 


for this analysis. 


ected automatically as Grouping Variables. No additional 


Click the Categorical Variables field in the Independent Variables section to activate it 
select the variable S TLANG as the independent variable. This is done by selecting S_ 


and 


TLANG from the list of available vari 


clic 


a drop-down menu will appear, f 
IEA IDB Analyzer will assume 


the 


ing the right arrow (>) but 


first 
With these settings, the intercep 
score for studen 
whereas the regression coefficient 


CT 


ava 


those students who spea 


anguage of the 
anguage of the test at home. 


Click the Plaus 


G 


ied) 


achievement 


ible Values radio butto 


ables and moving it to the Categorical Variables field by 


ton inthis section. By clicking the Contrast field of S TLANG, 


rom which Dummy Coding should be selected. By default, 
that S TLANG has two categories and it will select the 
tegory (code O) according to the coding of the variable as the reference category. 
t or constant will be the estimated average CT achievement 
ts with “other language[s]” spoken at home rather than the language of test, 
S_TLANG (estimate)) shows the estimated difference in 
score points between students speaking the language of test at home and 
another language at home. These settings also allow the user to 
perform a t-test to determine if the average CT achievement for students who speak the 
test at home differs significantly from that of students who do not speak the 


n in the Dependent Variable section and select 
PVCTO1-05 as the Plausible Values. This is done by selecting PVCTO1-05 from the list of 


ilable variables and moving it to the Plausible Values field by clicking the right arrow (>) 


button in this section. 


The Weight Variable is automatically selected by the software. As this example analysis 
uses student questionnaire data, TOTWGTS is selected by default. 


(10) Specify the name and the folder of the output files in the Output Files field by clicking the 


Define/Modify button. 
(11) Click the Start SPSS bu 


win 


tton to create the SPSS syntax file. The file will open in an SPSS syntax 


dow. The syntax file will be executed by opening the Run menu of SPSS and selecting the 
All option. If applicable, the IEA |DB Analyzer will prompt the user to confirm that they want 
to overwrite any existing files. 
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Figure 4.11: IEA IDB Analyzer setup for example student-level regression analysis with achievement scores 


HHT 18 Analyze: Analyse Module (lenion 403 a ig 


|] Anaysis Fite: | CACILSZOTE\Data\WerkBSEALLS av select 


Static Type 
dent Weighs) =] [linear Regression =] | 


Data Option, Number of Desir Z IEA 


2 


© Select Variables 


©) Grom Serbian, WZ Ech espa rom Analyse 


County ID - Alpha Code | 

asia ABOUT YOUhen were you born/Month U 
| 4 is26018 ABOUT YOUMWhen Ye 
} Proc ABOUT YOU/Are you a gi 
Sisacos [ABOUT YOU/Which oft 


| Boxcos YOURHOME AND YOUR FAMILY/n what 
Biszcoac YOURHOME AND YOUR FAMILY/In what 
‘ YOUR HOME AND YOUR FAMILY/What lz 


YOUR HOME AND YOUR FAMILY/At 
IS2G15AA YOUR HOME AND YOUR FAMILY/# 
Procisss YOUR HOME AND YOUR FAMILV/¥ 
Pisocise YOUR HOME AND YOUR FAMILY/In 
s2ct6a ‘YOUR USE OF ICT/How long have yt 
‘YOUR USE OF ICT/How i 


© Dependant Viable Nan Pausibe tue © Plauible Valuer 
Plasible Values 


| 'Nane Descipt 


= jAL PvcTot.os 1ST TO STHPV A 


“Gisci76 


Output files: | CAICILS2078\Data\a Nodi Returnto Main Menu |[ Help 
2) 


© [ statsess 


SPSS _ICILS (Using Student Weights) Linear Regression Use PVs ekaterinamikheeva 


The IEAIDB Analyzer outputs the mean CT achievement scores for each country for students who 
did (marked “Constant”) and students who did not (flagged by the variable S TLANG_D2) speak 
the language of the test at home (Figure 4.12). As an example, these results show that, in France, 
the estimated mean CT achievement of target-grade students speaking the language of test at 
home (Constant) was 445.64, with a standard error of 9.24. The French target-grade students 
speaking the language of test at home had an estimated mean CT achievement score 64.07 points 
(S_TLANG_D2) higher than French students who did not speak the language of test at home, with 
standard error of 9.46. The estimated t-test value is 6.93, which in absolute value is greater than 
1.96, indicating that this difference is statistically significant at a 95% confidence level. In the model 
S 
S 


tatistics section of the results, the output (Figure 4.12) shows that about 5% of the variance in 
tudent CT achievement in France can be attributed to whether the language the student took 
the ICILS 2018 test in was the language spoken at home. 
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4,12: IEA IDB Analyzer output for example student-level regression analysis with CT achievement scores 


Regression Coefficients 
Variable Regression Regression Regression Stndrdzd. Stndrdzd. Stndrdzd. 
Coefficient Coefficient Coefficient Coefficient Coefficient Coefficient 
IDCNTRY (s.e.) (t-value) (s.e.) (t-value) 
Denmark (CONSTANT) 480.98 8.78 54.75 . : ‘ 
S_TLANG_D2 50.13 8.96 5.59 Peli} -03 5.28 
Finland (CONSTANT) 474.10 10.01 47.38 ‘i ¢ a 
S_TLANG_D2 38.11 9.95 3.83 -10 -03 3.80 
France (CONSTANT) 445.64 9.24 48.21 ‘ ‘ : 
S_TLANG_D2 64.07 9.46 GofT o23: 103 6.93 
Germany (CONSTANT) 438.86 6.89 63.70 ‘ ; c 
S_TLANG_D2 63.37 6.93 9.14 225. -03 9.77 
Germany, (CONSTANT) 439.44 5.88 74.69 ‘i : ‘ 
North Rhine- S_TLANG_D2 64.25 6.68 9.61 28 03. 10.32 
Westphalia 
Korea, Republic of (CONSTANT) 480.48 34.01 14.13 ‘ ‘i : 
S_TLANG_D2 56.27 Bie Se 1.74 -04 03 1.69 
Luxembourg (CONSTANT) 451.60 1.12 403.24 “i : ; 
S_TLANG_D2 44.88 4.19 10.71 mays -02 10.70 
Portugal (CONSTANT) 489.06 10.52 46.49 3 ‘ : 
S_TLANG_D2 -7.44 10.13 “73 =.02 -03 =373) 
United States (CONSTANT) 468.15 Dail? 91.46 a : ‘ 
S_TLANG_D2 34.10 4.76 7.17 12 -02 6.55 
Table Average (CONSTANT) 463.15 4.50 102.94 3 é ; 
S_TLANG_D2 45.30 4.38 10.35 15 -O1 Ly 
Model Statistics 
R-Square R-Square Adjusted Adjusted 
(s.e.) R-Square R-Square 
IDCNTRY (s.e.) 
Denmark -02 -O1 -02 -O1 
Finland -O1 -O1 -O1 -O1 
France 05 02 05 02 
Germany -06 -O1 .06 -O1 
Germany, North .08 +102 .08 .02 
Rhine-Westphalia 
Korea, Republic of -00 .00 .00 .00 
Luxembourg -03 -O1 .03 .O1 
Portugal -00 -00 -00 -00 
United States -O1 -00 -O1 .00 
Table Average -03 -00 03. .00 


Note: s.e. = standard error. 
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4.4.4 Calculating percentages of students reaching proficiency levels 


This section describes the IEA |DB Analyzer can be used to perform benchmark analyses, which 
compute the percentages of students reaching specified proficiency levels on the CIL achievement 
scale and within specified subgroups, along with appropriate standard errors. 


As an example, we now describe how the IDB Analyzer can be used to compute the percentages 
of students (not) reaching the four ICILS 2018 international proficiency levels of CIL achievement 
(level 1 = 407 to 491 scale points; level 2 = 492 to 576 score points; level 3 = 577 to 661 score 
points; and level 4 = above 661 scale points) using the merged BSGALLI2.SAV data file. This 
analysis replicates results from the ICILS 2018 international report (Table 4.6; see Fraillon et al. 
2020a, table 3.5, p. 76). 


Table 4.6 : Percentages of students at each proficiency level of CIL achievement, originally published in the 
ICILS 2018 international report 


Country Below Level 1 Level 1 Level 2 Level 3 Level 4 Students achieving at each level (94) 

(less than 407 (from 407 to 491 | (from492to576 | (from 577 to 661 (above 661 0 10 20 30 40 50 60 70 80 90 100 

scale points) scale points) scale points) scale points) scale points) 
Chile 20 (1.7) 34 (1.3) 36 (1.6) 10 (1.0) O (0.1) es a 
Denmark* ! 3 (04) 14 (0.9) 45 (13) 36 (15) 3 (0.5) ——_ 
Finland 8 (0.9) 20 (1.1) 43 (1.3) 27 (1.4) 3 (04) | 
France 3 (1.0) 30 (1.2) 40 (1.2) 15 (1.0) 1 (0.2) ee 
Germany O (1.1) 23 (1.3) 43 (1.2) 22 (1.4) 2 (05) Se 
Kazakhstan* 54 (2.1) 27 (1.4) 415 (1.4) 4 (0.8) O (0.2) SSS 
Korea, Republic of 9 (07) 19 (1.2) 32 (1.4) 32 (1.6) 9 (0.8) ee ee ae 
Luxernbourg 9 (0.5) 32 (0.7) 38 (0.7) 11 (0.4) O (0.2) ee | 
Portugal! 7 (09) 27 (1.2) 46 (1.4) 19 (1.2) 1 (04) —_— 
Uruguay 33 (1.6) 29 (1.3) 27 (1.5) 9 (1.0) 1 (0.2) een ee 
ICILS 2018 average 8 (04) 25 (0.4) 36 (0.4) 19 (0.4) PaO.) M7 | 11 
Testing at the beginning of the school year 
Italy 24 (13) 39 (1.2) 30 (1.2) 7 (07) 0 (0.1) a 
Not meeting sample participation requirements 
United States 10 (06) 24 (08) 41 (07) 23 (08) 2 (03) _—————— | 
Benchmarking participants meeting sample participation requirements 
Moscow (Russian Federation) 3 (06) 45 (1.2) 44 (14) 34 (13) 3 (04) = ——— | 
North Rhine-Westphalia (Germany) 8 (1.2) 28 (1.4) 44 (15) 19 (1.1) 2 (04 ——_ eer 
Notes: Standard errors appear in parentheses. Because results are rounded to the nearest whole number, some totals may appear inconsistent. 
* Met guidelines for sampling participation rates only after replacement schools were included. MBelowLevell1 Mi Levell Hi Level2 
* Nearly met guidelines for sampling participation rates after replacement schools were included. BB Level 3 Level 4 


+ National defined population covers 90% to 95% of the national target population. 


Source: Fraillon et al. (2020a, table 3.5, p. 76). 


This example is performed by the analysis module of the IEA IDB Analyzer as follows (Figure 4.13 
shows how the analysis module should look when the correct settings for this example analysis 
are entered): 


Open the Analysis Module of the IEA IDB Analyzer. 

Specify the data file BSGALLI2.SAV as the Analysis File by clicking the Select button. 
Select ICILS (Using Student Weights) as the Analysis Type. 

Select Benchmarks as the Statistic Type. 


Om KR WN 


Select the Discrete option under the Benchmark Option drop-down menu to get discrete 
percentages of students reaching the ICILS 2018 international benchmarks. 


6) The variable IDCNTRY is selected automatically as Grouping Variables. No additional 
grouping variables are needed for this analysis. 


7) Specify the achievement scores to be used for the analysis by clicking the Plausible Values 
(8) field. Select PVCILO1-05 from the list of available variables and move it to the Plausible 
Values field by clicking the right arrow (>) button in this section. 
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(8) Specify the ICILS 2018 International Benchmarks, namely 407, 492, 576, and 661 (level 
1, level 2, level 3 and level 4, respectively). These values can be entered manually in the 
Achievement Benchmarks field, each separated by a blank space, or they can be selected by 
clicking on the drop-down menu available for this field. 


(9) The Weight Variable is automatically defined by the software. As this example analysis uses 
student questionnaire data, TOTWGTS is selected by default. 


(10) Specify the name and the folder of the output files in the Output Files field by clicking the 
Define/Modify button. 


(11) Click the Start SPSS button to create the SPSS syntax file. The file will open in an SPSS syntax 
window. The syntax file will be executed by opening the Run menu of SPSS and selecting the 
All option. If necessary, the IEA IDB Analyzer will prompt the user to confirm that they want 
to overwrite existing files. 


Figure 4.13: IDB Analyzer setup for example benchmark analysis 


[BE IA IDB Analyzer: Analysis Module - (Version 4.0.30) - a x 
@ Analysis File: | C:\\CILS2018\Data\Work Select 
(2) 
Analysis Type: Statistic Type: Plausible Value Option: Benchmark Option: Number of Decimals: Show Graphs l E 
ICILS (Using Student Weights) ~) [Benchmarks ©] Use Pvs Discrete ¥] fz. “Nes : A 


© Select Variables: 


© Grouping Variables: Exclude Missing From Analysis 


Name 


\of iDcnTRY Country ID - Numeric Code 3 


@ Plausible Values: [1] Report cases with no plausible values (Not classified) 


Country ID - Alpha Code 
ABOUT YOU/When were you born/Month 
ABOUT YOU/When were you born/Year 
ABOUT YOU/Are you a gir or a boy er — 
ABOUT YOUMWhich of the following [levels of education] do you ex... ‘| fino istiosue 
YOUR HOME AND YOUR FAMILY/In what country were you and you... 
YOUR HOME AND YOUR FAMILY/In what country were you and you... (yecierenen tecorete 
YOUR HOME AND YOUR FAMILY/In what country were you and you.. 
YOUR HOME AND YOUR FAMILY/What language do you speak at h.. Rall alae bani “Ci Compute percentages within benchmarks 
YOUR HOME AND YOUR FAMILY/Whatis the highest level of educat. G) Weight Variable: 
YOUR HOME AND YOUR FAMILY/What is the highest level of educet.. 
YOUR HOME AND YOUR FAMILY/About how many books are there i.. ee Name jescrip 
YOUR HOME AND YOUR FAMILY/How many of the following ICT de.. é 
YOUR HOME AND YOUR FAMILY/How many of the following ICT de.. 
YOUR HOME AND YOUR FAMILY/Intemet connection at home 
YOUR USE OF ICT/How long have you been using each of the follow.. 
YOUR USE OF ICT/How long have you been using each of the follow.. 
YOUR USE OF ICT/How long have you been using each of the fallow.. 
YOUR USE OF DIGITAL DEVICES/Who mainly taught you the followin... 

_ Biszci7e YOUR USE OF DIGITAL DEVICES/Who mainly taught you the followin.. 


© Output Files: C\ICILS2018\Data\analysis\Proficiency.* Modify Return to Main Menu Help 


SPSS _ICILS (Using Student Weights) Benchmarks Use PVs Discrete _ekaterina.mikheeva 


The IEA IDB Analyzer outputs the percentages of students reaching the specified benchmarks 
(Figure 4.14). In Germany, 9.69 percent of the target-grade students are below the proficiency 
level 1 score of 407 points (with a standard error of 1.12%), 23.47 percent of students reached 
proficiency level 1 (with a standard error of 1.27%), 42.93 percent of the students reached 
proficiency level 2 (with a standard error of 1.18%), 22.02 percent of students reached proficiency 
level 3 (with a standard error of 1.14%), and 1.89 percent of the students did score at proficiency 
level 4 (with standard error of 0.45%). 
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Figure 4.14: IEA IDB Analyzer output for example benchmark analysis of levels of CIL achievement 


Percents by Performance Groups of PVCIV 


Country ID - Performance Group N of Sum of Sum of Percent Percent 
Numeric Code Cases TOTWGTS TOTWGTS (s.e.) 
(s.e.) 
Chile 1.Below 407.001 604 44393 4157.15 19.69 1.72 
2.From 407.001 to Below 492.00 954 76793 3153.78 34.06 1.31 
3.From 492.001 to Below 576.00 1109 81610 3723.01 36.20 1.61 
4.From 576.001 to Below 661.00 403 21855 2230.08 9.69 96 
5.At or Above 661.00 21 803 316.16 36 -14 
Denmark 1.Below 407.001 64 1686 254.89 2.57 wad 
2.From 407.001 to Below 492.00 324 8957 620.17 13.63 .89 
3.From 492.001 to Below 576.00 1065 29286 1256.67 44.57 -30 
4.From 576.001 to Below 661.00 879 23746 1137.81 36.14 -50 
5.At or Above 661.00 72 2032 331.38 3.09 249 
Finland 1.Below 407.001 218 4522 545.13 7.76 .91 
2.From 407.001 to Below 492.00 311 11380 752.14 19.54 315 
3.From 492.001 to Below 576.00 1092 24821 922.35 42.61 135: 
4.From 576.001 to Below 661.00 663 15970 908.16 27.42 -42 
5.At or Above 661.00 62 1559 229:,21 2.68 ~39 
France 1.Below 407.001 394 04849 8562.32 13.07 -03 
2.From 407.001 to Below 492.00 923 244141 10180.30 30.44 .18 
3.From 492.001 to Below 576.00 1166 320956 11452.18 40.02 -24 
4.From 576.001 to Below 661.00 432 24027 7803.68 15.47 -98 
5.At or Above 661.00 25 7996 1968.28 1.00 22d 
Germany 1.Below 407.001 301 70813 8114.79 9.69 2 
2.From 407.001 to Below 492.00 911 71554 9497.04 23.47 wed 
3.From 492.001 to Below 576.00 1592 313722 9692.66 42.93 .18 
4.From 576.001 to Below 661.00 787 60915 9192.14 22.02 -14 
5.At or Above 661.00 65 13821 3335.22 13:89) 245 
Table average 1.Below 407.001 . : A 15..73 sail. 
2.From 407.001 to Below 492.00 ‘ ‘ : 25.76 31 
3.From 492.001 to Below 576.00 : i ‘ 37.41 ~34 
4.From 576.001 to Below 661.00 : 5 ; 19.09 -30 
5.At or Above 661.00 . 5 E 2.00 .10 


Notes: N = number, s.e. = standard error. The output shows the first five countries, and is for illustrative purposes only. 


4.4.5 Computing correlations with context or background variables and achievement 
scores 


In addition to the analyses described above, the IEA IDB Analyzer also is able to compute 
correlations between context or background variables, and between context or background 
variables and achievement scores. The next example is a correlation analysis with achievement 
scores. Correlation analysis between two context or background variables follows the same steps. 
The only difference is that the correlation between two context or background variables will require 
adding two variables in the Analysis Variables field instead of one. 


The examples of correlation between a context or background variable and CIL scores presented 
here cannot be found in the ICILS 2018 international report. This example calculates the 
correlation between students’ computer experience in years (S EXCOMP and the computer 
and information literacy (CIL) score (represented by the five plausible values (PVCILO1-05). The 
steps required to perform this correlation analysis are as follows (Figure 4.15 shows how the 
analysis module window should look when the correct settings for the analysis are entered): 


(1) Open the Analysis Module of the IEA IDB Analyzer. 
(2) Specify the data file BSGALLI2.SAV as the Analysis File. 


ANALYZING THE ICILS 2018 DATA USING THE IEA IDB ANALYZER 


Figure 4.15: IEA IDB analyzer setup for 


example correlation analysis 


HE eA 108 Analyzer: Analysis Module - (Version 40.30) 


@ Analysis File: €:\ Select 
2 
Analysis Type: Statistic Type: Plausible Value Option: Missing Data Option: ‘Number of Decimals: IEA 
ICILS (Using Student Weights) ¥ | [Correlations ¥ | (Use PVs ¥ | [Listwise ¥] [2 m7 


© Select Variables: 


©) Grouping Variables: Exclude Missing From Analysis 


Highest ISCED of parents 
Highest ISE! of parents 

Home literacy index 

Immigration status (dichotomous) 
Immigration status 

Smartphone experience in years 
Tablet experience in years 

ICT studies in current school year 
Internet access at home 


Learning of ICT tasks at school 
Negative perceptions of ICT for society 


ICT self-efficacy regarding the use of specialist applications 


ICT self-efficacy regarding the use of general applications 


Country ID - Numeric Code 


©) Analysis Variables: 


© Plausible Values: 


| PVCILOT-05 1ST TO 5TH PV 3 
© Weight Variable: 


|» 2 torwers Final Student Weight 3 


© Output Files: CAiCILS2018\Data\analysis\S_EXCOMP.” 


Modify Return to Main Menu Help 


© | start spss 


SPSS _ICILS (Using Student Weights) Correlations Use PVs _ekaterina.mikheeva 


o other variable needs to be 


field to activate it. 


field to activate it. Select S EX 
Analysis Variables field by clic 


7) Select PVCILO1-05 from the | 


Define/Modify button. 


window. The syntax file will be 


3) Select ICILS (Using Student Weights) as the Analysis Type. 
4) Select Correlations as the Statistic Type. The IDCNTRY (country ID) is selected by default. 


selected for this analysis. 


5) Specify the achievement scores to be used for the analysis by clicking the Plausible Values 


6) Specify the variable S EXCOMP as the analysis variable by clicking the Analysis Variables 


COMP from the list of available variables and move it to the 
king the right arrow (}) button in this section. 


ist of available variables and move it to the Plausible Values 


field by clicking the right arrow (») button in this section. 


8) The Weight Variable is automatically defined by the software. As this example analysis uses 
student questionnaire data, TOTWGTS is selected by default. 


9) Specify the name and the folder of the output files in the Output Files field by clicking the 


10) Click the Start SPSS button to create the SPSS syntax file. The file will open in an SPSS syntax 


executed by opening the Run menu of SPSS and selecting the 


All option. If necessary, the IEA IDB Analyzer will prompt the user to confirm that they want 


to overwrite existing files. 


The IEA IDB Analyzer outputs the c 
correlation between students’ com 
Uruguay is 0.36 with a standard er 
significant, the t-value is computed 
Using the exact numbers from the 

95% confidence level, there is a stat 
computer experience in years and s 


orrelation analysis in SPSS format (Figure 4.16) revealing the 
puter experience in years and the CIL achievement score in 
ror of 0.02. To verify whether this correlation is statistically 
by dividing the correlation coefficient by its standard error. 
Excel output gives a t-value of 18, which indicates that, on a 
istically significant correlation in Uruguay between students’ 
tudents’ CIL achievement. 
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Figure 4.16: !EA IDB Analyzer output for example correlation analysis 


Correlation Coefficients 


Correlation Correlation Correlation Correlation 
with with with with 
S_EXCOMP S_EXCOMP PV_CIL PV_CIL 
IDCNTRY Variable (s.e.) (s.e.) (s.e.) 
Chile S_EXCOMP 00 .00 29 02 
PV_CIL 29 02 -00 -00 
Denmark S_EXCOMP 00 .00 19 02 
PV_CIL .19 -02 -00 00 
Finland S_EXCOMP 00 00 22 02 
PV_CIL sed 02 .00 00 
France S_EXCOMP -00 00 abd 02 
PV_CIL -1l 02 -00 00 
Germany S_EXCOMP 00 00 12 03 
PV_CIL rae Be) 03 -00 -00 
Germany, North S_EXCOMP 00 00 12 03 
Rhine-Westphalia PV_CIL 22: .03 00 .00 
Italy S_EXCOMP 00 00 oak 02 
PV_CIL sail 02 -00 00 
Kazakhstan S_EXCOMP .00 00 34 02 
PV_CIL 34 02 00 00 
Korea, Republic of S_EXCOMP 00 00 129) 02 
PV_CIL .29 202 -00 00 
Luxembourg S_EXCOMP 00 .00 vA: .O1 
PV_CIL AO 4: -00 00 
Portugal S_EXCOMP 00 00 16 02 
PV_CIL .16 02 00 00 
Russian Federation S_EXCOMP 00 00 14 02 
(Moscow) PV_CIL ih 02 .00 00 
United States S_EXCOMP 00 00 “oy 02 
PV_CIL .27 «O02 -00 00 
Uruguay S_EXCOMP -00 00 -36 «02 
PV_CIL .36 02 00 00 
Table Average S_EXCOMP .00 .00 221 201 
PV_CIL sail O01 -00 00 


Note: s.e. = standard error. 


4.4.6 Calculating percentiles of students’ achievement 


The IEA IDB Analyzer can also be used to calculate percentiles of CIL (or CT) scores by selecting 
the Percentiles analysis option. This computes the percentiles within the distribution of student 
achievement scores within specified subgroups of students. This analysis type also computes the 
ppropriate standard errors for those percentiles. 


wo 


Inthe next example, we compute the 5th, 25th, 75th, and 95th percentiles of student CIL scores 
nd their standard errors within each country, using the weighting variable TOTWGTS, replicating 
values that were previously reported in the ICILS 2018 international report (Table 4.4; see Fraillon 
et al. 2020a, table 3.4, p. 75). The data are derived from the data file BSGALLI2.sav and the standard 
error calculations are based on replicate weights. 


ied) 


ANALYZING THE ICILS 2018 DATA USING THE IEA IDB ANALYZER 


To perform a percentile analysis, we again use the Analysis Module of the IEA IDB Analyzer as 


follows (Figure 4.17 shows how the analysis module should look when the correct settings for the 

example analysis are entered): 

1) Open the Analysis Module of the IEA IDB Analyzer. 

2) Specify the data file BSGALLI2.SAV as the Analysis File. 

3) Select ICILS (Using Student Weights) as the Analysis Type. 

4) Select Percentiles as the Statistic Type. 

5) The IDCNTRY (country ID) is selected by default as a grouping variable. No other grouping 
variable needs to be selected for this analysis. 

6) Specify the CIL achievement scores to be used for the analysis by clicking the Plausible 
Values field to activate it. Select PVCILO1-05 from the list of available variables and move it 
to the Plausible Values field by clicking the right arrow ()) button in this section. 

(7) The Weight Variable is automatically defined by the software. As this example analysis uses 
student questionnaire data, TOTWGTS is selected by default. 

(8) Click on the Percentiles radio button and specify the percentile points in the distribution. 
For this example we use the 5th, 25th, 75th and 95th percentiles. These need to be typed in 
increasing order separated by spaces. 

(9) Specify the name and the folder of the output files in the Output Files field by clicking the 


Define/Modify button. 


(10) Click the Start SPSS button to create the SPSS syntax flle. The file will open in an SPSS syntax 


window. The syntax file will be executed by opening the Run menu of SPSS and selecting the 
All option. If necessary, the IEA IDB Analyzer will prompt the user to confirm that they want 
to overwrite existing files. 


Figure 4.17: IEA IDB Analyzer Analysis Module setup screen for computing percentiles 


BEDIEA 1DB Analyzer: Analysis Module - (Version 4.0.30) 


oO Analysis File: | C:\ICILS2018\Data\Work\BSGALLI2. sav 


Select 


12) 
Analysis Type: 
ICILS (Using Student Weights) 


Statistic Type: 


» | [Percentiles » | [Use PVs 


Plausible Value Option: Number of Decimals: 
1 [2 
l 


@GIEA 


|| @& Select Variables: 


| A PVCTO1-05 1ST TO 5TH PV 


SPSS _ICILS (Using Student Weights) Percentiles Use PVs — ekaterinamikheeva 


(A) Grouping Variables: [M1 Exclude Missing From Analysis 


Name 


Description 
>of IDCNTRY Country ID - Numeric Code 3 


Name 


| |APVCiLO1-05 1ST TO STH PV 4 


G) Percentiles: 


[5.257595 


G) Weight Variable: 


Ls [>| 


Na 


me Description | 
> DtoTweTs Final Student Weight 3 
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The IEA IDB Analyzer outputs the percentile analysis in SPSS format (Figure 4.18). The example 
reveals that, in Portugal the score of the 5th percentile of the score distribution is 392.33 points 
(standard error 8.32), for the 25th percentile it is 472.04 points (standard error 4.00), for the 
75th percentile it is 564.75 points (standard error 3.73), and for the 95th percentile it is 622.81 
points (standard error 3.30). 


Figure 4.18: IEA IDB Analyzer SPSS output for percentiles 


Percentiles for PVCIV by IDCNTRY 

Country ID - N of Sum of p5 p5_se p25 p25_se p/5 p/5_se p95 p95_se 
Numeric Code Cases TOTWGTS 

Chile 3092 225453 325.19 11.36 425.23 6.22 534.12 3.46 599.98 265 
Denmark 2404 65708 434.56 5395 514.60 3.02 598.05 2.38 650.07 3.97 
Finland 2546 58252 381.40 7.61 486.00 4.19 586.67 3.562 645.27 4.03 
France 2940 801969 35:7..38 9.64 448.25 4.04 554.75 2.76 618.20 3.78 
Germany 3655 730825 371.73 8.95 469.77 4.69 573.99 2.50 633.82 4.28 
Italy 2810 541124 315.40 6.82 410.06 5.59 517.68 2.70 586.84 3.285) 
Kazakhstan 3371 224452 216.80 9.77 322.47 7.67 470.78 6.84 564.11 8.91 
Korea, Republic of 2875 439683 369.52 7.10 481.36 4.90 610.01 2.82 681.52 4.83 
Luxembourg 5401 6216 329.38 3.99 428.21 1.92 542.02 1.21 606.06 QLD 
Portugal 3221 99087 392.33 8.32 472.04 4.00 564.75 3738 622.81 3.30 
United States 6790 3856661 373.43 4.89 468.67 3.09 576.11 2.22 638.43 3.12 
Uruguay 2613 46323 279.62 6.82 379.53 52 526.72 5.28 605.33 7.08 
Russian Federation 2852 90584 425.23 7.25 509.15 2.80 DOS 22. 2.46 650.85 3.74 
(Moscow) 

Table Average 354.04 2.08 448.71 1.24 558.29 93 623.73 1.27 
Germany, 

North Rhine- 1991 164197 384.55 7.45 466.63 3.95) 265.013 3.24 628.95 3.78 
Westphalia 


Notes: N = number, s.e. = standard error, p5 = 5th percentile, p25 = 25th percentile, p75 = 75th percentile, p95 = 95th percentile. 


4.5 Performing analyses with teacher-level data 


As already 
sampling d 
presented 


here determines 


noted, student and teacher data cannot be merged and analyzed together due to the 
esign of ICILS 2018. The example of an analysis using teacher questionnaire data 
teachers’ average confidence in doing ICT tasks for teachers under 


and over 40 years of age. The analysis can, of course, be conducted only at the level of teachers. 


The analys 
also Fraillo 


teachers under 40 and teach 


net al. 2020a, tab 


is replicates results reported 


in the ICILS 2018 international report (Table 4.7; see 


e 6.3, p. 182). Note that this example simply computes the means of 
ers over 40 years of age. 
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Table 4.7: Example of teacher-level analysis, originally published in the ICILS 2018 international report 
Country National average scores of teacher confidence in doing ICT tasks 
All teachers 40 years and over Under 40 years Cuaron 40 45 50 55 20 
Chile 49 (0.3) Vv 46 (04) 50 (0.4) -4.0 (0.7) — = 
Denmarki * 53 (0.3) A 53 (0.4) 55 (0.4) -1.9 (0.6) eel 
Finland 51 (0.2) A 49 (0.3) 55 (0.3) -6.0 (0.4) = = 
Italy? 47 (0.3) v 46 (0.3) 51 (0.6) -4.7 (0.6) = == 
Kazakhstan" 47 (04) Vv 46 (0.5) 49 (0.5) -3.1 (0.6) = 
Korea, Republic of 50. (0.3) 49 (0.5) 53 (0.4) -3.8 (0.6) = = 
Portugal 53 (0.3) A 52 (0.3) 57 (0.4) -5.1 (0.5) — co 
ICILS 2018 average 50 (0.1) 49 (0.2) 53 (0.2) -4.1 (0.2) 
Not meeting teacher sample participation requirements 
France" 45 (0.3) 43 (0.3) 47 (0.4) -4.3 (0.5) = = 
Germany 45 (0.3) 44 (04) 48 (0.4) -44 (0.6) = = 
Luxembourg 47 (0.4) 45 (0.6) 49 (0.5) -4.1 (0.8) ——— i 
United States" 54 (0.3) 53 (0.5) 56 (0.4) -3.7 (0.7) =i = 
Uruguay 50 (0.3) 48 (0.5) 52 (04) -3.2 (0.6) =| m= 
Benchmarking participants meeting sample participation requirements 
Moscow (Russian Federation) 50 (0.3) A 50 (0.3) 52 (04) -2.5 (0.5) a 
North Rhine-Westphalia (Germany) 45 (0.3) v 43 (0.3) 48 (04) -5.1 (0.4) 1) = 


National ICILS 2018 results are: 

A More than three score points above average 
A Significantly above average 

V_ Significantly below average 

Vv More than three score points below average 


Notes: Standard errors appear in parentheses. Comparisons with ICILS 2018 only reported for countries or benchmarking 
participants meeting sample participation requirements. 


[BB Average score +/- confidence interval for teachers 
240 
/- confidence interval for teachers 


Onaverage across items, students with a score in the range with this 
colour have more than 50% probability to indicate: 


Never done this 


Know how to do this 


Statistically significant differences (p < 0.05) between subgroups are shown in bold. 

Met guidelines for sampling participation rates only after replacement schools were included. 
1 National defined population covers 90% to 95% of national target population. 
* Country surveyed target grade in the first half of the school year. 


Source: Fraillon et al. (2020a, table 6.3, p.182). 


As with previous examples, the first step is to identify the variables relevant to the analysis in 
the appropriate files, and review the documentation for any specific national adaptations to the 
questions of interest (see Appendix B of this user guide). The teacher questionnaire data files 
provide variable IT2GO2, which contains the age ranges for teachers: (i) under 25 years of age; (ii) 
25 to 29 years: (iii) 30 to 39 years; (iv) 40 to 49 years: (v) 50 to 59 years; and (vi) 60 years of age 
and over. Because these age-range categories are more detailed than required for this example, 
here they are collapsed into two categories (namely, under 40 years old and 40 years old and over). 
The SPSS syntax (see Figure 4.19) then loads the data file BTGALLI2.SAV into SPSS (see section 
4.2.4 for details) and completes all necessary recoding, thus creating anew variable. It then assigns 
labels to the new categories and saves the file. 


Figure 4.19: Example SPSS program to recode variable IT 2GOQ2 for the teacher-level analysis 


GET FILE = 'C:\ICILS2018\Data\Work\BTGALLI2.sav. 


RECODE IT2G02 (LOWEST THRU 3 = 0) (4 THRU 6 = 1) 
(ELSE = COPY) INTO IT2GO2col. 


VARIABLE LABELS IT2GO2col 'Collapsed IT2G02: About You/How old are you’. 


VALUE LABELS IT2GO2col 

O'Under 40' 

1'40 and over' 

8 'Not administered/missing by design’ 
9 'Presented but not answered‘ 


MISSING VALUES IT2GO2col (8, 9). 
FORMATS IT2GO02col (F 1.0). 


EXECUTE. 
SAVE OUTFILE = 'C:\ICILS2018\Data\Work\BTGALLI2.sav'. 
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Once this operation is complete, the next step is to reload the file into the IEA IDB Analyzer; the file 
can now be used for the analysis, and software set to recognize the newly-created variables. The 


An 


alyzer analysis module automatically selects the country variable (IDCNTRY) and the variables 


containing the sampling information used to compute the error estimates. 


The analysis module of the IEA IDB Analyzer is used to perform the example teacher-level 
analysis as follows (Figure 4.20 shows how the analysis module should look when the correct 


settings are entered): 


Open the Analysis Module of the IEA |DB Analyzer. 
Select the data file BTGALLI2.SAV as the Analysis File. 


Select ICILS (Using Teacher Weights) as the Analysis Type to analyze the responses of the 
teachers. 


Select Percentages and Means as the Statistic Type. 


Specify the variable IT2GO2col (collapsed teacher age ranges) as asecond grouping variable 
by clicking the Grouping Variables field to activate it. Select |T2GO2col from the list of 
available variables and move it to the Grouping Variables field by clicking the right arrow (> 
) button in this sect 
Locate and add the variable for the scale T_ICTEFF (“Teachers ICT self-efficacy”) as an 
Analysis Variable. (The ICILS 2018 technical report and Appendix C of this user guide 
provide more details on the scales and their construction.) 


on. 


The Weight Variable is automatically defined by the software. As this example analysis uses 
only teacher questionnaire data, TOTWGTT is selected by default. 


Specify the name and the folder of the output files in the Output Files field by clicking the 
Define/Modify button. 
Click the Start SPSS button to create the SPSS syntax file. The file will open in an SPSS syntax 
window. The syntax file will be executed by opening the Run menu of SPSS and selecting the 
All option. If necessary, the IEA |DB Analyzer will prompt the user to confirm that they want 
to overwrite existing files. 
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The IDB Analyzer outputs each country’s results on two lines, one for each value of the IT2GO2col 
variable (Figure 4.21). In this case, the selected variable has two categories (Under 40/40 and 
over). For categorical variables with more than two categories, the output will show one line 
per category for each country. The results are presented in the same manner as in the previous 
examples, revealing that, in Finland for example, the average of the scale “Teachers ICT self-efficacy” 
(T_ICTEFF) is 55.06 for teachers under 40 years of age and 49.09 for teachers of 40 years of age 
and more (standard errors are 0.33 and 0.29, respectively). 


Figure 4.20: IEA IDB Analyzer setup for example teacher-level analysis 


BE IADB Analyzer: Analysis Module - (Version 4.0.35) - o x 
@ Analysis File: { CA1CILS2018\Data\Work\ BTGALLI2.sav Seed 
3 ° 
alee Statistic Type: Plausible Value Optio) (Numberof Decanals Show Graphs 
[ICILS (Using Teacher Weights) _ | [Percentages and Means ~| [None Used =| (2 =] [es z 
© Select Variables: 


©) Grouping Variables: Exclude Missing From Analysis 


Sex of teacher 
ICT experience with ICT use during lessons 


ICT experience with ICT use for preparing lessons 
te Tables by: 
Teaching target grade schools in the current year © Separate Tabes by 


Collaboration between teachers in using ICT 
Negative views on using ICT in teaching and learning ‘| 
Positive views on using ICT in teaching and learning 


Teacher emphasis of teaching coding tasks in class © Analysis Variables: 
Emphasis on ICT capabilities in class 


Use of ICT for teaching practices in class 
Use of digital learning tools 

Use of general utility software 

Use of ICT for classroom activities 
Availability of computer resources at school 


BT ICTEFF ‘Teachers ICT self-etfiacy 3 


© Weight Variable: 


I 

Teacher participation in structured learning professional development r.. [> + ® sa Teaches Wg 5 
Teacher participation in reciprocal learning professional development re.. 

‘School weight adjustment - teacher study 

Teacher weight adjustment 

School base weight 

Teacher base weight 


© Output Files: | :\\CiLS2078\Data\Work\T_ICTEFF.* Modify Return to Main Menu Help 


SPSS _ICILS (Using Teacher Weights) Percentages and Means None Used _ekaterina.mikheeva 


69 


ICILS 2018 USER GUIDE FOR THE INTERNATIONAL DATABASE 


“UOIJEIADP P4AePueys = ‘AIC 'P}S ‘JOIIA PULPULJS = “d"S JAQLUNU = NY :S9ION 
Te SE° €S°8 ee OL°CY Os" 79°6S 81° 6eOT BE 9S7LT £88 Taso pue OF eTTeydisem-outyy 
7° bo Sé°L 6° BL°L7 Os* 9° OF TT'07¢8 LL°9LGS8T ELS 07 4epun yaIoN ‘AueuteED 
80° 97° 6 el” CS LP oy BL°T9 . : . JOeAO pue OF 
80° St°8 i TS*Ts Sv" eC BE . . : 07 aepup eBersae 9T qe], 
Lo (re 67°83 fo" 85°67 Le LE°99 cE OLY ce CESST 90ST FeAO pue QF (MOS Op) 
oT" 9e° Core BE° TT°¢s Lo” bor BE 66° 797 £0° T7208 BTL OF JFepul) uot BTepeyd UPTSSNy 
£6°2 L£e* €S*OT GS° 97°87 €L° S0°87 76° 99T VE ° 060% Sc9 4eAO pure 07 
TL° o Oc" LT°6 O07" OL°TS oy S6°TS Le°76T 79° CCUH £99 07 tepul Aensn3y 
€O°T 97° Sc°OT TS* 6S°CS 89° LO°9S 8° SOOT £0°7T968T SSLT TeAoO pue QF 
crt ce’ 71°6 9E° LO°9S 89° €6°€7 8° CS69 ST*T8S87T TeV 07 tepun seo eis perruf 
07" £7" 97° 6 Oe* 9L°TS 88" SE°78 L9O°SLY TE O7OET 68e7C TeAo pue QF 
oT" 8c" 69°9 77° T6°9S 88° S9°ST T7777 99° 7LCY Oc 07 tepul Tesniiog 
cS" 6G? 77° 6 79° LO°U7 70°C CC 97 Sé°OS 81° L726 Ef? 4eAQ pue 07 
Sc* Oe 07°8 TS* €9°87 70°C BL°eS 60° C7 7L°S8LOT €SZ 07 Jepuf) Banoquexn] 
ce" 96° 6¢°OT 97° 88°87 cee OL°T9 tC 697T TT * 89672 VIEL TeAO pue QF 
80° Tes €L°6 LE* S9°CS Go°¢ Of 8E TE VTL 87° 967ST c08 OY tepuf) Jo oTTqndey ‘eerz0y 
PL 67" T9° CT 7S° cO° 9% 9S°* 7B°ES 7S°CV7OC 07° 9970S OeVvT FeAO pue OF 
€8° 67° L9°ET 87° €T 67 9S" OT 97 L7°78ET L7°7OTE? BSTT 07 tepul ue Ts Yyyezey 
T6° €e- 6£°6 8° cv 97 00° 77°06 00° €6E7C T9°066S8 98ST Taso pue OF 
00° ce 99°'1 6S" 60°TS 00° 9S°6 08° 196 7L°7806 B89T 07 4epuf) AT eA 
esi 67° €L°8 am TS" ey 89° GE*E9 €¢2°966S T9°O0698ET (coal TeAo pue QF 
OT" 97° COL 77 06°L7 89° S9°9E S9°89E7 T78€c08 £88 OY tepuf) Aueurey 
EGS Lee St*6 Se" L6°C? £6" 6e°T9 60°ECCE €8°9LT76 T06 JeAo pue OF 
60° 8c" 60°8 Be LO°L £6" T9°8E ST’ SCTE 00°6¢76S cSS 07 tepul aoueig 
ia 8T° 6£°6 6e° 60°67 61° c8°L9 S6°67E €7 °O0660T LSZI Taso pue OF 
00° A SoTL ee 90°SS io ia BI te 06°0E7S eV STs 76S OF tepuf) pueTura 
G38" Ge" 97°8 cy 8S °CS G9" T6°99 GS" 8EE TS*OTLL BTL TeAO pue QF 
6S" Is° B9°L cy" OS°7S G9" 60° E 67ST? OO°ETSE O6E 07 tepuf) yaeuued 
c8" €c° TS*ZL 77° OT’ 97 C7? TL °8E S8°76ET 6€°S7C6T OEL FeAO pue OF 
(ace 0c Oc°L 07° OT*OS T7° 7 9¢°T9 99° LOTT 87° 8770€E 876 07 tepuf eT Tyo 
(728°8) 
SUTSSTW ('28°S) ("a"s) (ueoul) ("38"s) SLOMLOL SLOMLOL soseo nok e318 p[o Moy epog oTrTeunNyN 
qusoteg*Aed’p3s Aed* prs d4aL0T 1 daaL01T 1 qJusoted quscred Jo ung go wns JON :ZOOTLI pesdetTqToo - aI Aaqunop 


(TOOZODZLI AUYLNOGI) 4d AaLOIL 205 eBerzsay 


70 


SISAJDUD [aAa|-1a4Oda} a[dubxa JOJ }NdyNo JazAjoUuY FGI WII TZ ansl4 


ANALYZING THE ICILS 2018 DATA USING THE IEA IDB ANALYZER 


4.6 Performing analyses with school-level data augmented with 


school-leve 


| data 


When performing analyses with the merged school-level data, the data are analyzed to make 
statements about the number or percentages of students attending schools with a given 
characteristic, rather than about the number or percentages of schools with a given characteristic. 
This example of school level analysis focuses on the percentage of students enrolled at schools 
where ICT coordinators reported that the use of ICT for teaching and learning was hindered a lot 
or to some extent by insufficient pedagogical resources, replicating values that were previously 
reported in the ICILS 2018 international report (Table 4.8; see Fraillon et al. 2020a, table 6.9, p. 
191). 


Table 4.8: Example of school-level analysis, originally published in the ICILS 2018 international report 


Taal 


Country Percentage of students enrolled at school with: 
Insufficient ICT Insufficient time for Lack of effective Lack of an Lack of incentives for Insufficient 
skills among teachers teachers to prepare professional learning. effective online learning teachers to integrate pedagogical support 
lessons resources for teachers support platform ICT use in their for the use of ICT 
teaching 

Chile 68 (5.7) 52 (5.2) v 59 (5.3) 58 (5.0) A 57 (5.0) 49 (3.8) 
Denmarkt * 53 (4.9) v 81 (3.6) A 34 (4.3) v 24 (4.2) v 32 (4.2) v 34 (45) W 
Finland 84 (3.9) A 76 (3.7) A 75 (4.5) A 39 (4.7) 78 (4.3) A 59 (5.2) 
France 67 (4.3) 55 (4.6) Vv 62 (4.0) 35 (4.6) Vv 42 (4.3) v 42 (4.6) 
Germany 84 (3.4) A 85 (2.8) A 74 (3.8) A 65 (4.4) A 77 (4.3) A 68 (42) 
Italy? 81 (3.1) A 66 (4.1) 66 (4.4) 54 (4.2) A 83 (3.3) A 54 (4.3) 
Kazakhstan* 43 (4.0) v 50 (4.0) v 47 (4.1) v 51 (4.4) 38 (4.0) v 41 (36) V 
Korea, Republic of 25 (3.4) v 55 (4.7) 45 (4.1) v 45 (4.6) 46 (4.4) v 45 (4.5) 
Luxembourg, 72 (0.0) A 40 (0.1) v 51 (0.0) Vv 30 (0.0) v 59 (0.1) 51 (0.1) 
Portugaltt * 72 (3.7) A 75 (3.5) A 72 (4.2) A 37 (3.6) Vv 57 (4.3) 57 (4.2) 
Uruguay 67 (5.1) 68 (4.5) 63 (4.6) 49 (5.4) 57 (5.1) 51 (5.2) 
ICILS 2018 average 65 (1.2) 64 (1.2) Sy (GE) 44 (1.3) By (ib) 50) (3) 
Not meeting student sample participation requirements 
United States 47 (3.6) 43 (3.7) 43 (3.4) 29 (2.9) 38 (3.5) 29 (3.3) 
Benchmarking participants meeting sample participation requirements 
Moscow (Russian Federation) 37 (4.3) Vv 22 (3.3) Vv 19 (3.1) Vv 20 (3.4) Vv 27 (4.5) Vv 10 (2.3) Vv 
North Rhine-Westphalia (Germany) 92 (2.4) A 87 (2.5) A 76 (4.5) A 62 (4.2) A 80 (4.2) A 82 (3.9) A 


Notes: Standard errors appear in parentheses. Because results are rounded to the nearest whole number, some totals may appear inconsistent. 
Comparisons with ICILS 2018 only reported for countries or benchmarking participants meeting sample participation requirements. 


+ Met guidelines for sampling participation rates only after replacement schools were included. 
+ Nearly met guidelines for sampling participation rates after replacement schools were included. 


National defined population covers 90% to 95% of national target population. 


* Country surveyed target grade in the first half of the school year. 


Source: Fraillon et al. (2020a, table 6.9, p.191). 


t 


The variable of interest here is 12G13G (“Insufficient ICT s 
13G inthe ICT coordinator questionnaire (see Appendix A). 
he percentages of students where ICT coordinators rep 
earning was hindered either “a lot” or “to some extent” 
Thus, as in the previous example, the first and second categories of variable 112G13G need first 


to be collapsed into one using an SPSS syntax file (Figure 4.22). 


National ICILS 2018 results are: 
A More than 10 percentage points above average 


A Significantly above average 
VY Significantly below average 


Vv More than 10 percentage points below average 


ills among teachers”), namely question 
ote that this example simply computes 
orted the use of ICT for teaching and 
by insufficient pedagogical resources. 
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Figure 4.22: Example SPSS program to recode variable !12G13G for school-level analysis 


O'A 


RECODE 112G13G ( 
(ELSE = COPY) INTO I12G13Gcol. 


VARIABLE LABELS I12G13Gcol 'Collapsed 112G13G: ICT Support/Use of ICT hindered/Insufficient 
of ICT skills among teachers’. 


VALUE LABELS I12G13Gcol 


MISS 


lot or to some extent’ 


1'Very little or not at all’ 
8 'Not administered/missing by design’ 
9 'Presented but not answered’ 


NG VALUES I12G13Gcol (8, 9). 
FORMATS I12G13Gcol (F 1.0). 


EXECUTE. 
SAVE OUTFILE = 'C:\ICILS2018\Data\Work\BSG_BCGALLI2.sav. 


GET FILE = 'C:\ICILS2018\Data\Work\BSG_BCGALLI2.sav. 
LOWEST THRU 2 = 0) (8 THRU 4 = 1) 


Once this operation is complete, the next step is to reload the file into the IEA IDB Analyzer; the 
file can now be used for the analysis and software set to recognize the newly created variables. The 


Analyzer analysis module 


containing the sampling information used to compute the e 


Note 
(TOT 
exclu 


The analysis module of the IEA | 
as follows (Figure 4.23 shows how the analysis module should look when the correct settings are 


automatica 


that merging the school- and student-level data mea 
WGTS) and its replicate weights are included in the 
ded because they are no lo 


y selects the country variable (IDCNTRY) and the variables 


rror estimates. 


ns that only the total student weight 
merged file. The school weights are 


nger meanineful or interpretable. 


DB Analyzer is used to perform the example school-level analysis 


Select the merged data file BSG_BCGALLI2.SAV as the Analysis File. 
Select ICILS (Using Student Weights) as the Analysis Type to analyze school questionnaire 


Specify the variable 112G13Gcol as a second grouping variable by clicking the Grouping 
Variables field to activate it. Select I12G13Gcol from the list of available variables and move 
it to the Grouping Variables field by clicking the right arrow (») button in this section. 


The Weight Variable is automatically selected by the software; As this example analysis uses 


entered): 
1) Open the Analysis Module of the IEA IDB Analyzer. 
2 
3 
data as student attributes. 
4) Select Percentages Only as the Statistic Type. 
5 
(6) 
student questionnaire data as well as school questio 
level, TOTWGTS is selected by default. 
(7) Specify the name and the folder of the output files in 
Define/Modify button. 
(8) Click the Start SPSS button to create the SPSS syn 


nnaire data disaggregated to student 


the Output Files field by clicking the 


tax file and open it in a SPSS syntax 


window. The syntax file can be executed by opening th 
All option. If necessary, the IEA |DB Analyzer will pron 


to overwrite existing files. 


e Run menu of SPSS and selecting the 


npt the user to confirm that they want 
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Figure 4.23: 1DB Analyzer setup for example analysis with school-level data 


IEE cA 108 Analyzer: Analysis Module - (Version 4.0.35) 


@ Analysis File: | C\ICILS2018\Data\Work\BSG_BCGALLI2sav 


2 
Analysis Type: Statistic Type: Number of Decimals: Show Graphs 
[ICILS (Using Student Weights) ~] [Percentages only | [2 +] [es 


© Select Variables: 


© Grouping Veriables: Exclude Missing From Analysis 


Jame 


Sf IDCNTRY ‘Country ID - Numeric Code 4 
| Birzc136c01 Collapsed lI2G13G: ICT Support/Use of ICT hindered/Insufficie... |» 


ICT Support/Use of ICT hindered/Too few computers connected to th... 
ICT Support/Use of ICT hindered/Insufficient Internet bandwidth or sp... 
ICT Support/Use of ICT hindered/Not enough computers for instructi.. 
ICT Support/Use of ICT hindered/Lack of sufficiently powerful comput... 
ICT Support/Use of ICT hindered/Problems in maintaining ICT equipm... 
ICT Support/Use of ICT hindered/Not enough computer software 

ICT Support/Use of ICT hindered/Insufficient of ICT skills among teac... 

ICT Support/Use of ICT hindered/Insufficient time for teachers to pre... © Weight Variable: 
ICT Support/Use of ICT hindered/Lack of effective professional leamin... = 
ICT Support/Use of ICT hindered/Lack of an effective online learning s... 4 | | : wale ae = 
ICT Suppor/Use of ICT hindered/Lack of incentives for teachers to int. ae 
ICT Support/Use of ICT hindered/Restricted access to useful Internet r... 
ICT Support/Use of ICT hindered/Insufficient technical ICT support 

ICT Support/Use of ICT hindered/Insufficient pedagogical support for... 
ICT Support/Taught as standalone subject [computing, computer scie... 
ICT Support/Emphasis given on the following tasks/Develop algorithms 
ICT Support/Emphasis given on the following tasks/Write computer p.. 
ICT Support/Emphasis given on the following tasks/Evaluate compute... 
ICT Support/Emphasis given on the following tasks/Develop applicati... 
ICT Support/Emphasis given on the following tasks/Refine computer... 


7 


© Output Files: | C:\ICiLS2018\Data\Work\lI2G13Gcol.” 


SPSS _ICILS (Using Student Weights) Percentages only Use PVs _ekaterinamikheeva 


The IDB Analyzer outputs each country’s results on two lines, one for each value of the I12G13Gcol 
variable (Figure 4.24). In this case, the selected variable has two categories (a lot or to some 
extent/very little or not at all and over). For categorical variables with more than two categories, 
the output will show one line per category for each country. The results are presented in the same 
manner as in the previous examples, revealing that, for example, 68.05 percent of target-grade 
students in Chile were enrolled in schools where the ICT coordinator reported that the use of 
ICT in teaching and learning was hindered a lot or to some extent by insufficient ICT skills among 
teachers (standard error of this estimate 5.73). 
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4.7 Trend analyses 


Four of the countries that participated in ICILS 2013 also participated in ICILS 2018. Three of 
these countries met the necessary sample participation requirements within each cycle to allow 
valid comparisons of students’ CIL achievement across the two cycles. When performing trend 
analyses, users should note that the process of equating the tests across the two cycles of ICILS 
introduces some additional error into the calculation of any test statistic. Therefore, this additional 
error, termed the equating error, needs to be recognized and incorporated when calculating the 
standard error of any differences between results from different cycles. 


In order to estimate the standard error (SE) of a trend statistic, the mean and its standard error 
needs to be estimated for each cycle. The trend is the difference between the means. To estimate 
the standard error of the difference, the two standard errors of the means need to be combined 
with the equating error: 


SE(uucisig) ~ Micits13)) = | SEnicisig + SE Se2yicisi3+ EqErr? 


where SE(uiciisig) ~ Hucits13)) is the the standard error of the trend analysis, 1 can be any statistic 
in units on the equated ICILS scale (mean, percentile, gender difference, but not percentages) and 
SEuicinstg and SE,icis1g are the respective standard errors of this statistic from the two surveys. 
EgErr denotes the equating error that reflects the uncertainty in the link between both assessments, 
which was equal to 3.9 score points for the ClL scale. Please see also chapter 13 of the ICILS 2018 
technical report (Fraillon et al. 2020b), which includes a section on reporting of differences. 
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[INTRODUCTION TO THE SCHOOL PRINCIPAL QUESTIONNAIRE] 


Thank you for taking part inthe 2018 International Computer and Information Literacy Study. The 
purpose of this study is to examine across different countries the extent to which young people in 
[target grade] have developed computer and information literacy, which is defined as the ability to 
use Information and Communications Technology (ICT) to investigate, create, and communicate 
with others at home, school, the workplace and in society. 


In this questionnaire ICT can refer to: 
e desktop computers 

e notebook or laptop computers 

e netbook computers 

e tablet devices 


e [smartphones], except when being used for talk and text. 


In this questionnaire you will find questions about: 


e You and your use of ICT 
e Characteristics of your school 
e The application of ICT in teaching and learning at your school 


e Aspects of the management of ICT in your school. 


Please answer the questions with reference to [the school of the sampled students] as a whole. 


In answering the questions please refer to the following definition of ‘school’: A school is [to be 
country adapted]. 


For some of the questions you are asked to answer referring only to [target grade] students. 
Please look at the instructions given in each of the questions. 


We thank you for your effort and cooperation! 


APPENDIXA 


ABOUT YOU AND YOUR USE OF ICT 


Qi 


Q2 


Are you female or male? 


Female Male 


IP2G01 


How often do you use ICT for the following activities? 


(Please mark one choice in each row) 


Search for information on the Internet 
or a network maintained by education 
authorities for its schools 


Provide information about an educational 
issue through a website 


Look up records in a database (e.g. ina 
student information system) 


Maintain, organize and analyze data 
(e.g. with a spreadsheet or database) 


Prepare presentations 
Communicate with teachers in your school 
Communicate with education authorities 


Communicate with principals and senior 
staff in other schools 


Communicate with parents 


Work with a learning management system 
(e.g. [Moodle}) 


Use social media to communicate with the 
wider community about school-related 
activities 


Management of staff (e.g. scheduling, 
professional development) 


Preparing the curriculum 


School financial management 


Never 


Less than 
once a 
month 
but not 

every 
week 


At least 
once a 
month 
but not 
every 
day 


At least 
once a 
week 


Every 
day 


IP2G02A 


IP2G02B 


IP2G02C 


IP2G02D 


IP2GO02E 


IP2GO02F 


IP2G02G 


IP2G02H 


IP2G021 


IP2G02J 


IP2G02K 


IP2GO02L 


IP2G02M 


IP2GO02N 
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YOUR SCHOOL 


When answering the questions in this section, please refer to the definition of ‘school’ given in the note at 
the beginning of this questionnaire. 


Q3 What is the total number of boys and girls in the school? 
Record 0 (zero), if none. 


Total number of girls Total number of boys 


IP2GO03A IP2G03B 


Q4 What is the total number of boys and girls in the school? 
Record 0 (zero), if none. 


Total number of girls Total number of boys 


IP2GO04A IP2G04B 


Q5 (a) What is the lowest (youngest) grade that is taught at your school? 


(Please mark only one choice) IP2GO5A 


[National Adaptation 1] 


[National Adaptation 2] 


[National Adaptation 3] 


[National Adaptation 4] 


[National Adaptation 5] 


[National Adaptation 6] 


[National Adaptation 7] 


[National Adaptation 8] 


b) What is the highest (oldest) grade that is taught at your school? 


(Please mark only one choice) IP2G05B 


[National Adaptation 9] 


[National Adaptation 10] 


[National Adaptation 11] 


[National Adaptation 12] 


[National Adaptation 13] 


[National Adaptation 15] 


APPENDIXA 


Q6 


Q7 


Q8a 


Q8b 


What are the total numbers of full-time and part-time teachers in your school? 
A tull-time teacher is employed on a regular basis as a teacher for at least 90% of full-time hours for the 
full school year. All other teachers should be considered part-time. 
Record 0 (zero), if none. 
a) Total number of full-time teachers IP2GO6A 
b) Total number of part-time teachers IP2G06B 
Which of the following best describes where your school is located? 
(Please mark only one choice) IP2GO7 
In a community with fewer than 3,000 people 
In a town with at least 3,000 but less than 15,000 people 
In a town with at least 15,000 but less than 100,000 people 
In a city with at least 100,000 but less than 1,000,000 people 
In a city with 1,000,000 or more people 
Is this school a public or a private school? 
(Please mark only one choice) IP2GO8A 
A public school 
(This is a school managed directly or indirectly by a public education authority, 
government agency, or governing board, appointed by government or elected 
by public franchise.) 
A private school 
(This is a school managed directly or indirectly by a non-government organization; 
for example, a church, trade union, business, or other private institution.) 
Approximately what percentage of students in your school have the following backgrounds? 
(Please mark one choice in each row) 
Oto 11 to 26 to More than 
10% 25% 50% 50% 
Come from economically affluent homes IP2GO08BA 


or a network maintained by education 
authorities for its schools 


Come from economically disadvantaged homes |P2GO08BB 
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ICT AND TEACHING IN YOUR SCHOOL 


Q9 ~~ Howimportant is each of the following outcomes of education in your school? 
(Please mark one choice in each row) 


Very Quite Somewhat Not 
important important important important 


a) The development of students’ basic computer IP2GO9A 
skills (e.g. internet use, email, word processing, 
presentation software) 


b) The development of students’ skills in using ICT IP2GO09B 
for collaboration with others 


c) The use of ICT for facilitating students’ IP2GO09C 
responsibility for their own learning 


d) The use of ICT to augment and improve IP2GO9D 
students’ learning 


e) The development of students’ understanding IP2GO9E 
and skills relating to safe and appropriate use 
of ICT 

f) |The development of students’ proficiency in |P2GO9F 


accessing and using information with ICT 


g) The development of students’ ability to write IP2G09G 
[apps] or programs 


APPENDIXA 


Q10 Are there procedures in place to monitor whether teachers at this school use ICT to achieve the 


following learning outcomes? 


(Please select all that apply for each row) 


The development of students’ 
basic computer skills (e.g. internet 
use, email, word processing, 
presentation software) 


The development of students’ skills 
in using ICT for collaboration with 
others 


The use of ICT for facilitating 
students’ responsibility for their own 
learning 


The use of ICT to augment and 
improve students’ learning 


The development of students’ 
understanding and skills relating to 
safe and appropriate use of ICT 


The development of students’ 
proficiency in accessing and using 
information with ICT 


The development of students’ ability 
to write [apps] or programs 
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Yes, by Yes, Yes, Yes, by No, 

reviewing through through other this is not 
lesson plans teacher observing means monitored 

self- classrooms 
evaluation 

IP2G10AA IP2G10AB IP2G10AC IP2G10AD IP2G10AE 
IP2G10BA IP2G10BB IP2G10BC IP2G10BD IP2G10BE 
IP2G10CA IP2G10CB IP2G10CC IP2G10CD IP2G10CE 
IP2G10DA IP2G10DB IP2G10DC IP2G10DD IP2G10DE 
IP2G10EA IP2G10EB IP2G10EC IP2G10ED IP2G10EE 
IP2G10FA IP2G10FB IP2G10FC IP2G10FD IP2G10FE 
IP2G10GA IP2G10GB IP2G10GC. IP2G10GD IP2G10GE 
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Q11 Are teachers in your school expected to acquire knowledge and skills in each of the following 
activities? 


(Please mark one choice in each row) 


Expected Expected Not 
and but not expected 
required required 
a) Integrate Web-based learning in their IP2G11A 
instructional practice 
b) Use ICT-based forms of student assessment IP2G11B 
c) Use ICT for monitoring student progress IP2G11C 
d) Collaborate with other teachers via ICT IP2G11D 
e) Communicate with parents via ICT IP2G11E 
f) | Communicate with students via ICT IP2G11F 
g) Integrate ICT into teaching and learning IP2G11G 
h) Use subject-specific digital learning resources IP2G11H 
(e.g. tutorials, simulation) 
i) | Use e-portfolios for assessment P2G111 
j) | Use ICT to develop authentic (real-life) assignments IP2G11J 
for students 
k) Assess students’ [computer and information literacy] IP2G11K 


APPENDIXA 


MANAGEMENT OF ICT IN YOUR SCHOOL 


Q12 Who has the main responsibility for making decisions about each of the following 
aspects of ICT in this school? 


(Please mark one choice in each row) 


[Ministry, School School [Heads ICT Information Individual No one 
department boards/ principal of coordinator specialist teachers 

or local/ councils or deputy department] or librarian 

district 
authority] 


a) The procurement of 
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IP2G12A 


ICT equipment 


b) The choice of non-digital 


IP2G12B 


earning materials 


c) The choice of digital 


IP2G12C 


earning materials 


d) The selection of a 


IP2G12D 


earning management 
system 


e) The maintenance of 


IP2G12E 


ICT equipment 


f) | Decisions about whether 


IBZAG AG 


ICT is used in teaching 


g) The implementation of 


IP2G12G 


ICT-based approaches 
in teaching 


h) |The implementation of 


IP2G12H 


ICT-based approaches 
in administration 


i) | The use of ICT-based 


IP2G12! 


approaches to 
assessment 


j) |The assessment of 


IP2G12J 


students’ [computer and 
information literacy] 


k) The implementation 


IP2G12K 


of a [computer and 
information literacy] 
curriculum at the school 


1) The development of 


IP2G12L 


ICT-related teacher 
competencies 


Q13 Does your school or school system have policies with regard to the following aspects of ICT use? 


(Please mark one choice in each row) 


The provision of security measures to prevent unauthorized 
system access or entry 


Restrictions on the number of hours students are allowed to sit at 
a computer 


Student access to school computers outside class hours 
(but during school hours) 


Student access to school computers outside school hours 


The fulfillment of intellectual property rights (e.g. software copyrights) 


Prohibitions of access to inappropriate material (e.g. pornography, 
violence) 


Student use of non-school related games on schoo! computers 


The provision of access to school computers and/or the Internet 
for the local community (parents and/or others) 


Support for [students with special needs or specific learning 
difficulties] 


Unacceptable behaviours towards other students (e.g. [Cyberbullying]) 


The provision of laptop computers and/or other mobile learning 
devices for student use at school and at home 


Student use of their own ICT at school 


Yes 


IP2G13A 


IP2G13B 


IP2G13C 


IP2G13D 


IP2G13E 


IP2G13F 


IP2G13G 


IP2G13H 


IP2G131 


IP2G13J 


IP2G13K 


IP2G13L 
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Q14_ Throughout the current school year, how many teachers in this school participate in the following 
forms of professional development about ICT for teaching and learning? 


(Please mark one choice in each row) 


None or Some of Most of — All or nearly 


hardly any them them all 

a) Courses on the use of ICT in teaching provided IP2G14A 
by the school or the school system 

b) Training by another teacher who has attended IP2G14B 
a course on ICT 

c) Discussions about the use of ICT in education IP2G14C 
as a regular item during meetings of the 
teaching staff 

d) Observations of colleagues using ICT in their IP2G14D 
teaching 

e) Group discussions of teachers about their IP2G14E 
use of ICT in teaching 

f) Participation in professional learning programs IP2G14F 
delivered online 

g) Participation in courses on ICT conducted by IP2G14G 
an external agency or expert 

h) — Participation in a [community of practice] IP2G14H 


concerned with ICT in teaching 
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Q15 At your school, what priority is given to the following ways of facilitating the use of ICT in teaching 
and learning? 


(Please mark one choice in each row) 


High Medium Low Nota 
priority priority priority priority 

a) Increasing the numbers of computers per IP2G15A 
student in the school 

b) Increasing the number of computers connected IP2G15B 
to the Internet 

c) Increasing the bandwidth of Internet access for IP2G15C 
the computers connected to the Internet 

d) Increasing the range of digital learning resources IP2G15D 
available for teaching and learning 

e) Establishing or enhancing an online learning IP2G15E 
support platform 

f) | Supporting participation in professional IP2G15F 
development on pedagogical use of ICT 

g) Increasing the availability of qualified technical IP2G15G 
personnel to support the use of ICT 

h) Providing teachers with incentives to integrate IP2G15H 
ICT use in their teaching 

i) Providing more time for teachers to prepare IP2G151 
lessons in which ICT is used 

j) _ Increasing the professional learning resources IP2G15J 


for teachers in the use of ICT 
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Section 2: ICT coordinator questionnaire 
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Information Literacy Study 


ICT-Coordinator Questionnaire for the Main Survey 
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[INTRODUCTION FOR ICT-COORDINATORS TO THE QUESTIONNAIRE] 


This questionnaire is concerned with Information and Communication Technology (ICT) in schools and 
in particular the resources and support available for its use. 


In this questionnaire ICT can be: 

e desktop computers 

e notebook or laptop computers 

e netbook computers 

e tablet devices 

e [smartphones], except when being used for talk and text. 


In this questionnaire you will find questions about: 
e Your position as ICT coordinator 

e ICT resources in your school 

e Support for ICT use in your school. 


Please answer as accurately as you can. 
We have estimated that it will take less than 15 minutes of your time to complete the questionnaire. 


Thank you for making that time available. 


APPENDIXA 


ABOUT YOUR POSITION 
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This questionnaire asks for information about ICT resources (including computers) in your school as well as 
pedagogical practices that use ICT. It is important that the person responding knows about the ICT facilities in 


your school and about practices regarding their use. 


The questionnaire should be completed by the person with designated responsibility for ICT in the school. If there 
is no person with designated responsibility for ICT in the school, the questionnaire should be completed by the 


principal or [deputy-principal]. 


If you do not have the information to answer particular questions, then please consult other persons in your 


school 


Qi Do you, at your school, hold the position of technology or computer coordinator? 


(Please mark only one choice) 


Yes, | formally serve as coordinator. 


Yes, | informally serve as coordinator. 


Q2__ Which of the following teaching duties do you have? 
(Please mark one choice in each row) 


a) | teach ICT courses to students. 


b) | teach other subjects (not related to learning about ICT) 
to students. 


C) | do not have any teaching duties for students. 


d) | teach ICT courses to, or conduct workshops for, teachers and 
other school staff. 


Q3 How many years has your school been using ICT for teaching and/or learning purposes 


for students in [target grade]? 


(Please mark only one choice) 


Never, we do not use ICT 


Fewer than 5 years 


At least 5 but fewer than 10 years 


10 years or more 


Yes 


No 


2GO01 


| am not the ICT-coordinator, but | am answering as the school principal or his/her designate. 


II2GO2A 


12G02B 


I2G02C 


112G02D 


Il2G03 
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ICT RESOURCES 


Q4 __ Please indicate the availability of the following technology resources in your school. 
(Please mark one choice in each row) 


Available to Available — Available Not 
teachers only to only to available 


and students teachers — students 


a) Digital learning resources that can be Il2GO4A 
accessed offline 


b) Digital learning resources that can only l2G04B 
be used online 


c) Access to the Internet through the school \12G04C 
network 
d) Access to an education site or network \l2G04D 


maintained by education authorities 


e) Email accounts for school-related use ll2G04E 


APPENDIXA 


Q5 


Please indicate the availability of each of the following software resources at your school. 


(Please mark one choice in each row) 


Practice programs or [apps] where teachers 
decide which questions are asked of students 
(e.g. [Quizlet, Kahoot], [mathfessor]) 


Single user digital learning games 
(e.g. [languages online]) 


Multi-user digital learning games with graphics 
and inquiry tasks (e.g. [Quest Atlantis) 


Word-processor software 
(e.g. [Microsoft Word®)) 


Presentation software 
(e.g. [Microsoft PowerPoint®)) 


Video and photo software for capture and 
editing (e.g. [Windows Movie Maker, iMovie, 
Adobe Photoshop)) 


Concept mapping software (e.g. [Inspiration®], 
[Webspiration®}) 


Data logging and monitoring tools 
(e.g. [Logger Pro]) that capture real-world data 
digitally for analysis (e.g. speed, temperature) 


Simulations and modelling software 
(e.g. [NetLogo}) 


A learning management system 
(e.g. [Edmodo], [Blackboard}) 


Graphing or drawing software 
e-portfolios (e.g. [VoiceThread)) 
Digital contents linked with textbooks 


Social media (e.g. [Facebook, Twitter]) 


Available to Available 
teachers only to 
and students teachers 


Available 


only to 
students 


Not 
available 


II2GOS5A 


I12G05B 


I2GO05C 


I12GO05D 


Il2GO5E 


I2GO5F 


Il2GO05G 


Il2GO5H 


Il2GO5| 


Il2GO5J 


lI2GO5K 


I2GO5L 


II2GO05M 


II2GOSN 
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Q6__—sC—vP lease indicate the availability of the following technology facilities at [target grade]. 
(Please mark one choice in each row) 


Available to Available — Available Not 
teachers only to only to available 


and students teachers students 


a) Remote access to a school network \I2GO6A 

b) Space on a school network to store files l2GO6B 

c) Aschool intranet with applications and \I2GO6C 
workspaces 

d) Internet-based applications for collaborative \I2GO6D 


work (e.g. [Google Docs®)) 


e) A83D printer ll2GO6E 
f) | Robots or robotic devices \l2GO6F 
g) Access to a wireless LAN (Wi-fi) lI2G06G 
h) A learning management system l12GO6H 


(e.g. [WebCT®}, [Moodle]) 
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Q7a 


In your school, approximately how many of the following types of (school-provided) ICT devices 
are available? 


For this question please: 


Count terminals (if they have a keyboard and a screen) as computers 
Exclude computers that are not in use (e.g. in storage) 

Exclude computers that are only used as servers 

Record 0 (zero), if none. 


All devices in the school Devices available for student use 
Desktop computers 
Il2GO7AA1 Il2GO7AA2 
Laptops/notebooks 
l2GO7AB1 \l2G07AB2 
Tablet devices 
I2GO7AC1 ll2GO7AC2 


Q7b Approximately, what percentage of all ICT devices in the school (include all types) are connected 
to the Internet? 


Q7c 


Q8 


% 112G07B 


In your school, about how many (school-provided) smart boards or interactive whiteboards are 
available? 


Record 0 (zero), if none. 


% l2G07C 


Does your school or [educational authority] provide teachers with their own portable digital 
device? 


(Please mark only one choice) 12G08 


Yes, for every teacher 


Yes, but not for all teachers 


No 
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Q9 Approximately what percentage of students at the [target grade] have access to portable 
computers (laptops, netbooks or tablet devices) at school? 


(Please mark one choice in each row) 


Oto 26 to 51 to 76 to 
25% 50% 75% 100% 
a) Students provided with portable computers ll2GO9A 
by their [school] for use at school only 
b) Students provided with portable computers ll2G09B 
by their [school] for use at home and at school 
c) Students bring portable computers which they \l2G09C 


own to use at school 


Q10 Where are school ICT devices for teaching and learning in [target grade] located? 
(Please mark one choice in each row) 


Yes No 
a) In most (80% or more) classrooms Il2G10A 
b) In computer laboratories 12G10B 
C) As class sets of computers that can be moved between \l2G10C 
classrooms 
d) In the library \l2G10D 
e) In other places accessible to students (e.g. cafeteria, auditorium, I2G10E 
study area) 
f) Students bring the devices to class I12G10F 
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ICT SUPPORT 


Q11_ At your school, who provides [routine/day-to-day] technical ICT support? 
(Please mark one choice in each row) 


Yes No 
a) Yourself l2G11A 
b) A network administrator in the school (other than yourself) 2G11B 
C) ICT technical staff (other than yourself) at the school Il2G11C 
d) Administrators or school staff (other than yourself) || 112G11D 
e) Teachers (other than yourself) N2G11E 
f) Staff from the relevant education authority responsible for 2G141F 
the school 
g) Personnel from external companies contracted to provide ll2G11G 
maintenance 
h) Students from this school 2G11H 


Q12_ At your school, who provides [routine/day-to-day] pedagogical ICT support for teachers? 
(Please mark one choice in each row) 


Yes No 
a) Yourself Il2G12A 
b) ICT technical staff (other than yourself) at the school 112G12B 
C) Administrators or school staff (other than yourself) 1l2G12C 
d) Librarians, library staff or information specialists (other than yourself) l2G12D 
e) Teachers (other than yourself l2G12E 
f) Staff from the relevant education authority responsible for the school 2G12F 


Q13. To what extent is the use of ICT in teaching and learning at your school hindered by each of the 


following obstacles? 
(Please mark one choice in each row) 


a) Too few computers with an Internet connection 
b) — Insufficient Internet bandwidth or speed 

c) Not enough computers for instruction 

d) Lack of sufficiently powerful computers 

e) Problems in maintaining ICT equipment 

f) | Not enough computer software 

g) Insufficient ICT skills among teachers 


h) — Insufficient time for teachers to prepare 
lessons 


i) — Insufficient effective professional learning 
resources for teachers 


j) | Lack of an effective online learning support 
platform 


k) — Insufficient incentives for teachers to integrate 
ICT use in their teaching 


) Restricted access to useful Internet resources 
m) Insufficient technical ICT support 


n) — Insufficient pedagogical support for the use 
of ICT 


To some 


extent 


I2G13A 


12G13B 


Il2G13C 


2G13D 


I2G13E 


I2G13F 


2G13G 


I2G13H 


I2G131 


I2G13J 


I2G13K 


I2G13L 


2G13M 


II2G13N 
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Q14 


Q15 


Is [computing, computer science, information technology, informatics or similar] 
taught as a standalone subject at the [target grade] in your school? 


Yes Please continue with question 15 


No Please go to the end of the questionnaire 
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12G14 


In the teaching of [computing, computer science, information technology, informatics or similar] at 


the [target grade] in your school, how much emphasis is given to the following tasks? 


(Please mark one choice in each row) 


Develop algorithms (e.g. instructions for 
a program like [Scratch]) 


Write computer programs 

Evaluate computer programs 

Develop applications 

Refine computer code to improve efficiency 
Debug computer code 

Develop simulations 

Test solutions to problems using simulations 


Create visual displays of processes 
(e.g. flow charts, decision trees) 


Strong 
emphasis 


Some 
emphasis 


Little 
emphasis 


No 
emphasis 


I2G15A 


112G15B 


I2G15C 


112G15D 


I2G15E 


N2G15F 


I2G15G 


I2G15H 


I2G151 
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Section 3: Teacher questionnaire 
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[INTRODUCTION FOR TEACHERS TO THE QUESTIONNAIRE] 


This questionnaire is concerned with Information and Communication Technology (ICT) in schools, 
its use in teaching and learning, and students’ development of Computer and Information 
Literacy (CIL). 

In this questionnaire ICT can refer to: 

e desktop computers 

e notebook or laptop computers 

e netbook computers 

e tablet devices 

e [smartphones], except when being used for talk and text. 


In this questionnaire you will find questions about: 

e Your background and familiarity with ICT 

e Your use of ICT in teaching a reference [target grade] class 
e The use of ICT in the school 

e Learning to use ICT in teaching. 


Some questions focus on a nominated ‘reference’ class. This is the first [target grade] class that 
you teach for a regular subject (i.e. other than home room, assembly) on or after the Tuesday 
following the last weekend before you first accessed this questionnaire. You may, of course, 
teach the class at other times during the week as well. 


If you did not teach a [target grade] class on that Tuesday, please use the [target grade] class 
that you taught on the first day after that Tuesday. 


Please answer as accurately as you can. You will mostly answer by clicking on a button. You 
can change your responses at any time until you have clicked on ‘I’ve finished’ at the end of 
the questionnaire. 


We have estimated that it will take less than 30 minutes of your time to complete the questionnaire. 


Thank you for making that time available. 
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ABOUT YOU 


Qi 


Q2 


Are you female or male? 


Female 


How old are you? 
Less than 25 
25-29 

30-39 

40-49 

50-59 


60 or over 


Male 


IT2G01 


IT2G02 
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Q3 
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What are the main subjects that you teach in this school in the current school year? 
(Please indicate the subjects that you teach in this school, indicating only those that individually account 
for at least [four lessons] each week in this school. The exact name of one or more of your subjects may 
not appear in the list for each category. If it does not, please mark the category you think best fits the 
subject.) 
[Language arts: test language] IT2G03A 
[Language arts: foreign or other national languages] IT2GO03B 
Mathematics [Add any appropriate national examples] IT2GO38C 
Sciences (general science and/or physics, chemistry, biology, geology, IT2G03D 


earth sciences, technical science) 


Human sciences / Humanities / Social Studies (history, geography, civics, IT2GO03E 
law, economics, etc.) 


Creative arts (visual arts, music, dance, drama, etc.) IT2G03F 
[Information technology, computer studies or similar] IT2G038G 
Practical and vocational subjects [Add any appropriate national examples] IT2GO3H 
Other (e.g. [moral/ethics, physical education, personal and social development)) IT2G03I 


In the current school year, at how many schools do you teach [target grade] students? 
(Please mark only one choice) 


IT2G04 


Only in this school 


In this and another school 


In this and in two other schools 


In this and in three or more other schools 
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ABOUT YOUR POSITION 


Q5 Approximately how long have you been using ICT for teaching purposes? 
(Please mark one choice in each row) 


Never Less than Between More than 
two years twoand _ five years 
five years 
a) During lessons IT2G05A 
b) Preparing lessons IT2GO5B 


Q6 _ How often do you use ICT in these settings? 
(Please mark one choice in each row) 


Never Lessthan Atleast Atleast Every 
oncea oncea_ oncea day 
month amonth aweek 

but not — but not 
every every day 


week 
a) At school when teaching IT2GO6A 
b) At school for other work-related purposes IT2GO6B 
c) Outside school for work-related purposes IT2GO06C 
d) Outside school for non-work-related purposes IT2GO6D 
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Q7 — How well can you do these tasks using ICT? 
(Please mark one choice in each row) 


| know how | haven’t | do not 
to do this done this | think | could 
but | could do this 
find out how 
a) Find useful teaching resources on the Internet IT2GO7A 
b) Contribute to a discussion forum / user group T2G07B 
on the Internet (e.g. a wiki or blog) 
c) Produce presentations (e.g. [Microsoft PowerPoint®] IT2GO7C 
or a similar program) with simple animation functions 
d) Use the Internet for online purchases and payments IT2GO7D 
e) Prepare lessons that involve the use of ICT by students IT2GO7E 
f) Use aspreadsheet program (e.g. [Microsoft Excel®)]) T2G07F 
for keeping records or analyzing data 
g) Assess student learning IT2G07G 
h) Collaborate with others using shared resources IT2G07H 
such as [Google Docs®], [Padlet] 
i) Use alearning management system (e.g. [Moodle], IT2G071 


[Blackboard], [Edmodo]) 
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YOUR USE OF ICT IN TEACHING 


In this section of the questionnaire please focus your responses on your teaching practices in a ‘reference’ 
class. 

This is the first [target grade] class that you teach for a regular subject (i.e. other than home room, assembly) 
on or after Tuesday following the last weekend before you first accessed this questionnaire. You may, of course, 
teach the class at other times during the week as well. If you did not teach a [target grade] class on that 
Tuesday, please use the [target grade] class that you taught on the first day after that Tuesday. 


Q8 Inthe current school year, at how many schools do you teach [target grade] students? 
(Please mark only one choice) 


IT2G08 


[Language arts: test language] 


[Language arts: foreign or other national languages] 


Mathematics [Add any appropriate national examples] 


Sciences (general science and/or physics, chemistry, biology, ge-ology, 
earth sciences, technical sciences) 


Human sciences / Humanities / Social Studies (history, geography, 
civics, law, economics, etc.) 


Creative arts (visual arts, music, dance, drama, etc.) 


[Information technology, computer studies or similar] 


Practical and vocational subjects [Add any appropriate national 
examples] 


Other (e.g. [moral/ethics, physical education, personal and social 
development) 
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Q9 si In your teaching of the reference class in this school year, how much emphasis have you given to 
developing the following ICT-based capabilities in your students? 


(Please mark one choice in each row) 


Strong Some Little No 
emphasis emphasis emphasis emphasis 


a) To access information efficiently IT2GO9A 

b) To display information for a given audience/ IT2G09B 
purpose 

c) To evaluate the credibility of digital information IT2GO9C 

d) To share digital information with others IT2GO09D 

e) To use computer software to construct digital IT2GO9E 


work products (e.g. presentations, documents, 
images and diagrams) 


f) To provide digital feedback on the work of others IT2GO9F 
(such as classmates) 


g) To explore a range of digital resources when IT2G09G 
searching for information 


h) To provide references for digital information IT2GO9H 
sources 
i) | To understand the consequences of making IT2G09| 


information publically available online 
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Q10 How often do students in your reference class use ICT for the following activities? 
(Please mark one choice in each row) 


They They They They They 
do not never sometimes often always 
engage use ICT uselCT uselCT uselCT 

in this in this in this in this in this 


activity activity activity activity —_ activity 


a) Work on extended projects IT2G10A 
(i.e. lasting over a week) 

b) Work on short assignments T2G10B 
(i.e. within one week) 

c) Explain and discuss ideas with other T2G10C 
students 

d) Submit completed work for assessment IT2G10D 

e) Work individually on learning materials at 1T2G10E 


their own pace 


f) | Undertake open-ended investigations or T2G10F 
field work 
g) Reflect on their learning experiences IT2G10G 


(e.g. by using a learning log) 


h) Communicate with other students on T2G10H 
projects 

i) | Plan a sequence of learning activities for T2G101 
themselves 

j) Analyze data IT2G10J 

k) Evaluate information resulting from a search IT2G10K 

) Collect data for a project IT2G10L 

m) Create visual products or videos IT2G10M 

n) Share products with other students IT2G10N 
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Q11 How often do you use ICT with the following practices when teaching your reference class? 
(Please mark one choice in each row) 


| do not | never They They They 
use this use ICT use ICT uselCT uselCT 
practice with this with this with this with this 


with the practice practice practice practice 
reference 
class 

a) The presentation of information through T2G11A 
direct class instruction 

b) The provision of remedial or enrichment 1T2G11B 
support to individual students or small 
groups of students 

c) The support of student-led whole-class IT2G11C 
discussions and presentations 

d) The assessment of students’ learning T2G11D 
through tests 

e) The provision of feedback to students on IT2G11E 
their work 

f) The reinforcement of learning of skills through T2G11F 
repetition of examples 

g) The support of collaboration among students T2G11G 

h) The mediation of communication between T2G11H 
students and experts or external mentors 

i) | The communication with parents or T2G11H 
[guardians] about students’ learning 

j) | The support of inquiry learning T2G11J 
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Q12 How often did you use the following tools in your teaching of the ref-erence class this school 


year? 
(Please mark one choice in each row) 


Practice programs or apps where you ask 
students questions (e.g. [Quizlet, Kahoot], 
[mathfessor]) 


Digital learning games 


Word-processor software 
(e.g. [Microsoft Word®)) 


Presentation software 
(e.g. [Microsoft PowerPoint®)) 


Spreadsheets (e.g. [Microsoft Ex-cel®)]) 


Video and photo software for capture and editing 
(e.g. [Windows Movie Maker, iMovie, Adobe 
Photoshop) 


Concept mapping software (e.g. [Inspiration®], 
[Webspiration®)]) 


Simulations and modelling software 
(e.g. [NetLogo}) 


A learning management system 
(e.g. [Edmodo], [Blackboard)}) 


Communication software 
(e.g. email, direct messaging, Skype) 


Collaborative software (e.g. [Google Docs®], 
[Onenote], [Padlet}) 


Computer-based information resources 
(e.g. topic-related websites, wikis, encyclopaedia) 


Interactive digital learning resources 
(e.g. learning objects) 


Graphing or drawing software 
e-portfolios (e.g. [Voice Thread)) 
Digital contents linked with textbooks 


Social media (e.g. [Facebook, Twitter]) 


Never 


In some 
lessons 


In most 
lessons 


In every 
or almost 


every 


iesson 


IT2G12A 


IT2G12B 


IT2G12C 


IT2G12D 


T2G12E 


T2G12F 


IT2G12G 


IT2G12H 


IT2G121 


IT2G12J 


IT2G12K 


T2G12L 


IT2G12M 


IT2G12N 


IT2G120 


IT2G12P 


IT2G12Q 
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Q13 In your teaching of the reference class this school year, how much emphasis have you given to 
teaching the following skills? 


(Please mark one choice in each row) 


Strong Some Little (e) 
emphasis emphasis emphasis emphasis 


a) To display information in different ways IT2G13A 
b) To break a complex process into smaller parts 1T2G13B 
C) To understand diagrams that describe or show IT2G13C 


real-world problems 


d) To plan tasks by setting out the steps needed to IT2G13D 
complete them 

e) To use tools to make diagrams that help solve IT2G13E 
problems 

f) To use simulations to help understand or solve IT2G13F 


real-world problems 


g) To make flow diagrams to show the different IT2G13G 
parts of a process 


h) To record and evaluate data to understand and T2G13H 
solve a problem 


i) To use real-world data to review and revise IT2G13I 
solutions to problems 
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IN YOUR SCHOOL 


Q14_ To what extent do you agree or disagree with the following statements about the use of ICT in 
teaching at your school? 


(Please mark one choice in each row) 


Strongly Agree Disagree Strongly 


agree disagree 

a) ICT is considered a priority for use in teaching. IT2G14A 

b) My school has sufficient ICT equipment T2G14B 
(e.g. computers). 

C) The computer equipment in my school is IT2G14C 
up-to-date. 

d) My school has access to sufficient digital learning T2G14D 
resources (e.g. learning software or [apps)). 

e My school has good connectivity (e.g. fast speed IT2G14E 
and stable) to the Internet. 

f) There is enough time to prepare lessons that IT2G14F 
incorporate ICT. 

g) There is sufficient opportunity for me to develop IT2G14G 
expertise in ICT. 

h) There is sufficient technical support to maintain T2G14H 


ICT resources. 


Q15_ To what extent do you agree or disagree with the following statements about your use of ICT in 
teaching and learning at your school? 


(Please mark one choice in each row) 


Strongly Agree Disagree Strongly 


agree disagree 

a) | work together with other teachers on improving IT2G15A 
the use of ICT in classroom teaching. 

b) | collaborate with colleagues to develop IT2G15B 
ICT-based lessons. 

C) | observe how other teachers use ICT in teaching. IT2G15C 

d) | discuss with other teachers how to use ICT in IT2G15D 
teaching topics. 

e) | share ICT-based resources with other teachers IT2G15E 


in my school. 
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LEARNING TO USE ICT IN TEACHING 


Q16 


Q17 


Did your [initial teacher education] include the following elements? 


(Please mark one choice in each row) 


Learning how to use ICT 


Learning how to use ICT in teaching 


Yes 


No 


IT2G16A 


IT2G16B 


How often have you participated in any of the following professional learning activities in the past 


two years? 
(Please mark one choice in each row) 


A course on ICT applications (e.g. word processing, 
presentations, internet use, spreadsheets, databases) 


A course or webinar on integrating ICT into teaching and 
learning 


Training on subject-specific digital teaching and learning 
reSOUurces 


Observations of other teachers using ICT in teaching 


An |CT-mediated discussion or forum on teaching and 
learning 


The sharing of digital teaching and learning resources with 
others through a collaborative workspace 


Use of a collaborative workspace to jointly evaluate 
student work 


A course on use of ICT for [students with special needs 
or specific learning difficulties] 


A course on how to use ICT to support personalized 
learning by students 


Not at all 


Once only More than 


once 


IT2G17A 


IT2G17B 


T2G17C 


IT2G17D 


IT2G17E 


IT2G17F 


IT2G17G 


T2G17H 


T2G171 
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APPROACHES TO TEACHING 


Q18_ To what extent do you agree or disagree with the following statements about using ICT in 
teaching and learning at school? 


(Please mark one choice in each row) 


Using ICT at school... Strongly Agree Disagree — Strongly 
agree disagree 
a) Impedes concept formation by students. IT2G18A 
b) Helps students develop greater interest in learning. T2G18B 
C) Helps students to work at a level appropriate to IT2G18C 
their learning needs. 
d) Results in students copying material from Internet T2G18D 
SOUICES. 
e) Helps students develop problem solving skills. IT2G18E 
f) Distracts students from learning. T2G18F 
g) Results in poorer written expression among IT2G18G 
students. 
h) Results in poorer calculation and estimation skills 1T2G18H 
among students. 
i) Limits the amount of personal communication IT2G18I 
among students. 
j) Enables students to collaborate more effectively. IT2G18J 
k) Helps students develop skills in planning and 1T2G18K 
self-regulation of their work. 
l) Improves academic performance of students. T2G18L 
m) Enables students to access better sources of T2G18M 


information. 
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[INTRODUCTION FOR STUDENTS TO THE QUESTIONNAIRE] 


This questionnaire is about Information and Communication Technology (ICT). 


In this questionnaire ICT can refer to: 

e desktop computers 

e notebook or laptop computers 

e netbook computers 

e tablet devices 

e [smartphones], except when being used for talk and text. 


In this questionnaire you will find questions about: 
e You, your home and your family 

e Where and hor often you use of ICT 

e What you use ICT for 

e Your views about the use of ICT. 


Please read each question carefully and answer as accurately as you can. In this questionnaire, 
you will mostly answer by clicking on a button. You can change your responses at any time until 
you have clicked on ‘I’ve finished’ at the end of the questionnaire. 


There are also a few questions where you will need to write a short response. 


In this questionnaire, there are no right or wrong answers. Your answers should be 
the ones that apply to you. 


You may ask for help if you do not understand something or if you are not sure how to answer 
a question. 


All your answers will be kept confidential. 
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ABOUT YOU 


Q1. When were you born? 
January — December (Month) 


1997 — 2008 (Year) 


Q2 Are youagirl or a boy? 


Girl Boy 


IS2GO1A 


IS2G01B 


IS2G02 


Q3 What is the highest level of education you expect to complete? 


(Please mark one choice) 


[ISCED level 6, 7 or 8] 
[ISCED level 4 or 5] 
[ISCED level 3] 
[ISCED level 2] 


| do not expect to complete [ISCED level 2] 


IS2G03 
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ABOUT YOU 


In this section you will be asked some questions about your family and your home. 


Some of these questions will be about home and your parents or guardians who look after you — for example, 
step-parents or foster-parents. Select one parent or guardian as [parent or guardian 1] and the other as [parent 


or guardian 2]. 


If you share your time with more than one set of parents or guardians, please answer the following questions 
for those parents/guardians with whom you spend the most time. If you share your time only with one parent, 


please answer only the following questions for this parent. 


Q4 In what country were you and your parents born? 
(Please mark only one choice in each column) 


IS2GO04A 


You 


[Country of test] 


[Other country/Group A] 


[Other country/Group B] 


[Another country] 


Q5 —_—~What language do you speak at home most of the time? 
(Please mark one choice) 


[Language of test] 


[Other language 1] 


[Other language 2] 


[Another language] 


Q6 __—Does your [parent or guardian 1] work in a paid [job]? 
(Please mark one choice) 


Yes |_| (Note: Student will be directed to Q7a and Q8a) 


No |__| (Note Student will be directed to Q7b and Q8b) 


IS2G04B 


[Parent or 


guardian 1] 


IS2GO04C 


IS2G05 


IS2G06 


[Parent or 


guardian 2] 
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Q7a 


Q8a 


Q7b 


Q8b 


What is your [parent or guardian 1]’s main [job]? 
(for example high school teacher, kitchen-hand, sales manager) 
(Please write in the [job] title) 
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IS2GO7A 


What does your [parent or guardian 1] do in his/her main [job]? 


(for example teaches high school students, helps the cook prepare meals in a restaurant, 
manages a sales team) 


(Please use a sentence to describe the kind of work he/she does in that [job]) 


IS2GO8A 


What was your [parent or guardian 1]’s last main [job]? 
(for example high school teacher, kitchen-hand, sales manager) 


Please tell us his/her last main [job]. If he/she has never had a paid [job], 
please write what he/she is currently doing. 


(Please write in the [job] title) 


IS2G07B 


What did your [parent or guardian 1] do in his/her last main [job]? 
(for example taught high school students, helped the cook prepare meals in a restaurant, 
managed a sales team) 


(Please use a sentence to describe the kind of work he/she did in that [job] or what 
he/she is currently doing if he/she has never had a paid [job]) 


IS2G08B 


126 


Q9g 


Q10 
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What is the highest level of education completed by your [parent or guardian 1]? 
If you are not sure which box to choose, please ask the [test administrator] for help. 


(Please mark only one choice) 


IS2GO09 


[ISCED level 6, 7 or 8] 

[ISCED level 4 or 5] 

[ISCED level 3] 

[ISCED level 2] 

He/she did not complete [ISCED level 2]. 


Does your [parent or guardian 2] work in a paid [job]? 
(Please mark one choice) 


IS2G10 


Yes (Note: Student will be directed to Q11a and Q12a) 
No |_| (Note: Student will be directed to Q11b and Q12b) 
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Qli1la 


Q12a 


Q11b 


Q12b 


What is your [parent or guardian 2]’s main [job]? 
(for example high school teacher, kitchen-hand, sales manager) 
(Please write in the [job] title) 
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IS2G11A 


What does your [parent or guardian 2] do in his/her main [job]? 


(for example teaches high school students, helps the cook prepare meals in a restaurant, 
manages a sales team) 


(Please use a sentence to describe the kind of work he/she does in that [job]) 


IS2G12A 


What was your [parent or guardian 2]’s last main [job]? 
(for example high school teacher, kitchen-hand, sales manager) 


Please tell us his/her last main [job]. If he/she has never had a paid [job], 
please write what he/she is currently doing. 


(Please write in the [job] title) 


IS2G11B 


What did your [parent or guardian 2] do in his/her last main [job]? 
(for example taught high school students, helped the cook prepare meals in a restaurant, 
managed a sales team) 


(Please use a sentence to describe the kind of work he/she did in that [job] or what 
he/she is currently doing if he/she has never had a paid [job]) 


IS2G12B 
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Q13 What is the highest level of education completed by your [parent or guardian 2]? 


Q14 


If you are not sure which box to choose, please ask the [test administrator] for help. 


(Please mark only one choice) 


[ISCED level 6, 7 or 8] 

[ISCED level 4 or 5] 

[ISCED level 3] 

[ISCED level 2] 

He/she did not complete [ISCED level 2]. 


About how many books are there in your home? 
(Please mark one choice) 


None or very few (0-10 books) 

Enough to fill one shelf (11-25 books) 

Enough to fill one bookcase (26-100 books) 
Enough to fill two bookcases (101-200 books) 


Enough to fill three or more bookcases 
(more than 200 books) 


IS2G13 


IS2G14 
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Q15a How many of the following ICT devices are currently used in your home? 

(Please mark one choice in each row) 

None One Two Three 
or more 

a) Desktop or [laptop] computers IS2G15AA 
b) Tablet devices or e-readers IS2G15AB 

(e.g. [iPad, Tablet PC, Kindle}) 
Q15b Do you have an Internet connection at home? 

(Please mark one choice) 

IS2G15B 


Yes 


No 
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YOUR USE OF ICT 


Q16 How long have you been using each of the following types of ICT devices? 
(Please mark one choice in each row) 


Never At least At least At least Seven 


or less one year three five years years 
than but less years but but less or more 
one than less than 
year three than seven 
years five years years 
a) Desktop or [laptop] computers IS2G16A 
b) Tablet devices or e-readers |IS2G16B 
(e.g. [iPad, Tablet PC, Kindle}) 
c) [Smartphones] except for using text IS2G16C 


and calling 


Q17 Who mainly taught you how to do the following activities? 
(Please mark one choice in each row) 


My ly ly I taught | have never 

teachers family friends myself — learned this 
a) Communicate over the Internet IS2G17A 
b) Create or edit digital documents |IS2G17B 
c) Create or edit digital presentations IS2G17C 
d) Change settings on an ICT device IS2G17D 
e) Find information on the Internet IS2G17E 
f) | Use programs and files in a computer IS2G17F 


network 
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Q18 How often do you use ICT in these places? 


Please do not count the use of [smartphones] when making phone calls or writing text messages. 


(Please mark one choice in each row) 


a) At school for school-related purposes 


b) At school for other purposes 


c) Outside of school for school-related purposes 


d) Outside of school for other purposes 


Q19 How often do you use ICT for each of the following activities? 
(Please mark one choice in each row) 


a) Write or edit documents 


b) Use a spreadsheet to do calculations, 


store data or plot graphs 
(e.g. using [Microsoft Excel®)]) 


c) Create a simple “slideshow” presentation 


(e.g. using [Microsoft PowerPoint®)) 


d) Record or edit videos 


e) Write computer programs, scripts or apps 


(e.g. using [Logo, LUA or Scratch) 


f) | Use drawing, painting or graphics software 


or [apps] 


g) Produce or edit music 


Never Lessthan Atleast At least Every 
once a oncea oncea day 
month month week 
but not — but not 
every every 
day 
IS2G18A 
IS2G18B 
IS2G18C 
IS2G18D 
Never Lessthan Atleast Atleast Every 
once a oncea  oncea day 
month month week 
but not — but not 
every every 
day 
IS2G19A 
IS2G19B 
IS2G19C 
IS2G19D 
IS2G19E 
IS2G19F 
IS2G19G 
IS2G19H 


h) Build or edit a webpage 
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Q20 How often do you use ICT to do each of the following communication activities? 


(Please mark one choice in each row) 


Never Less than Atleast At least Every 
oncea oncea_ oncea day 
month month week 

but not — but not 
every every 


day 

a) Share news about current events on social IS2G20A 
media 

b) Communicate with friends, family, or other IS2G20B 
people using instant messaging, voice or 
video chat (e.g. [Skype, WhatsApp, Viber]) 

c) Send texts or instant messages to friends, IS2G20C 
family, or other people 

d) Write posts and updates about what happens IS2G20D 
in your life on social media 

e) Ask questions on forums or [Q&A, question IS2G20E 
and answer] websites 

f) | Answer other peoples’ questions on forums |S2G20F 
or [Q&A, question and answer] websites 

g) Write posts for your own blog IS2G20G 
(e.g. (WordPress, Tumblr, Blogger]) 

h) Post images or video in social networks or IS2G20H 
online communities (e.g. [Facebook, 
Instagram or YouTube]) 

i) | Watch videos or images that other people IS2G20I 
have posted online 

j) | Send or forward information about events or IS2G20J 


activities to other people 
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Q21 


How often do you use ICT to do each of the following leisure activities? 


(Please mark one choice in each row) 


Search the Internet to find information about 
places to go or activities to do 


Read reviews on the Internet of things you 
might want to buy 


Read news stories on the Internet 


Search for online information about things 
you are interested in 


Use websites, forums, or online videos to find 
out how to do something 


Play games 
Listen to downloaded or streamed music 


Watch downloaded or streamed TV shows 
or movies 


Never 


Less than 
once a 
month 


Atleast At least 
oncea oncea 
month week 
but not —_ but not 
every every 
day 


Every 
day 
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IS2G21A 


IS2G21B 


IS2G21C 


IS2G21D 


IS2G21E 


IS2G21F 


IS2G21G 


IS2G21H 
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YOUR USE OF ICT 


Q22 How often do you use ICT for the following school-related purposes? 


(Please mark one choice in each row) 


Never Less than Atleast Atleast Every 
oncea oncea_ oncea day 
month month week 

but not — but not 
every every 


day 

a) Prepare reports or essays IS2G22A 
b) Prepare presentations IS2G22B 
c) Work online with other students IS2G22C 
d) Complete [worksheets] or exercises IS2G22D 
e) Organize your time and work IS2G22E 
f) Take tests IS2G22F 
g) Use software or applications to learn skills IS2G22G 

or a subject (e.g. mathematics tutoring 

software, language learning software) 
h) Use the Internet to do research IS2G22H 
i) Use coding software to complete IS2G22I 


assignments (e.g. [Scratch]) 


j) | Make video or audio productions IS2G22J 
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Q23 At school, how often do you use ICT during lessons in the following subjects or subject areas? 
(Please mark one choice in each row) 


| don't Never Insome_ Inmost_ In every 
study this lessons lessons or 
subject/ almost 
these every 
subjects lesson 
a) [Language arts: test language] IS2G23A 
b) [Language arts: foreign or other national IS2G23B 
languages] 
c) Mathematics [Add any appropriate national IS2G23C 
examples] 
d) Sciences (general science and/or physics, IS2G23D 
chemistry, biology, geology, earth sciences) 
e) Human sciences /Humanities/Social studies IS2G23E 
(history, geography, civics, law, economics, etc.) 
f) Creative arts ([visual] arts, music, dance, IS2G23F 
drama, etc.) 
g) _ [Information technology, computer studies IS2G23G 
or similar] 
h) Practical or vocational [Add any appropriate IS2G23H 
national examples] 
i) — Other (e.g. [moral/ethics, physical education, IS2G23| 


personal and social development) 
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When studying throughout this school year, how often did you use the following tools during 
class? 


(Please mark one choice in each row) 


Never In some In most In every 
lessons lessons or almost 
every 
lesson 
Tutorial software or [practice programs] IS2G24A 
Word-processing software (e.g. [Microsoft Word®)) IS2G24B 
Presentation software (e.g. [Microsoft IS2G24C 
PowerPoint®)) 
Spreadsheets (e.g. [Microsoft Excel®]) IS2G24D 
Multimedia production tools (e.g. media capture IS2G24E 
and editing, web production) 
Concept mapping software (e.g. [Inspiration®], IS2G24F 
[Webspiration®)]) 
Tools that capture real-world data (e.g. speed, IS2G24G 
temperature) digitally for analysis 
Simulations and modelling software IS2G24H 
Computer-based information resources IS2G24l 
(e.g. websites, wikis, encyclopaedia) 
Interactive digital learning resources IS2G24J 
(e.g. learning games or applications) 
Graphing or drawing software IS2G24K 
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Q25_ At school, to what extent have you learned how to do the following tasks? 


Q26 


(Please mark one choice in each row) 


Provide references to Internet sources 
Search for information using ICT 


Present information for a given audience or 
purpose using ICT 


Work out whether to trust information from 
the Internet 


Decide what information obtained from the Internet 
is relevant to include in school work 


Organize information obtained from Internet 
sources 


Decide where to look for information on the Internet 
about an unfamiliar topic 


Use ICT to collaborate with others 


At school, have you learned about the importance of the fol-lowing topics? 


(Please mark one choice in each row) 


To change passwords regularly (e.g. network account, email, 


social media) 


Toa 
large 
extent 


Toa 
moderate 
extent 


Toa 
sma 


extent 


ot at 
all 
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IS2G25A 


IS2G25B 


IS2G25C 


IS2G25D 


IS2G25E 


IS2G25F 


IS2G25G 


IS2G25H 


To check the origin of emails before opening attachments 


To log out of a shared computer at the end of a session 


To share information on social media responsibly 


Yes 


No 


IS2G26A 


IS2G26B 


IS2G26C 


IS2G26D 
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YOUR THOUGHTS ABOUT USING AND LEARNING ABOUT ICT 


Q27 How well can you do each of these tasks when using ICT? 
(Please mark one choice in each row) 


| know | have | do not 
how to do never done think | 
do this this but | could do 
could work this 
out how to 
do this 

a) Edit digital photographs or other graphic images IS2G27A 
b) Create a database (e.g. using [Microsoft Access®}) IS2G27B 
c) Write or edit text for a school assignment IS2G27C 
d) Search for and find relevant information for a school IS2G27D 

project on the Internet 
e) Build or edit a webpage IS2G27E 
f) | Change the settings on your device to im-prove the |\S2G27F 

way it operates 
g) Create a computer program, macro, or [app] IS2G27G 

(e.g. in [Basic, Visual Basic]) 
h) Set up a local area network of computers or other ICT IS2G27H 
i) Create a multi-media presentation (with sound, IS2G27| 

pictures, or video) 
j) | Upload text, images, or video to an online profile IS2G27J 
k) Insert an image into a document or message IS2G27K 
) — Install a program or [app] IS2G27L 
m) Judge whether you can trust information you find IS2G27M 


on the Internet 
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Q28 How much do you agree or disagree with the following statements about ICT? 
(Please mark one choice in each row) 


Strongly Agree Disagree Strong 
agree disagree 

a) Advances in ICT usually improve people’s living IS2G28A 

conditions. 
b) ICT helps us to understand the world better. IS2G28B 
C) Using ICT makes people more isolated in society. IS2G28C 
d) With more ICT there will be fewer jobs. IS2G28D 
e People spend far too much time using ICT. IS2G28E 
f) ICT is valuable to society. IS2G28F 
g) Advances in ICT bring many social benefits. IS2G28G 
h) Using ICT may be dangerous for people’s health. IS2G28H 
i) | would like to study subjects related to ICT after IS2G28| 

[secondary school]. 
j) | hope to find a job that involves advanced ICT. IS2G28J 
k) Learning how to use ICT applications will help me IS2G28K 


to do the work | am interested in. 
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Q29 When studying during the current school year, to what extent have you been taught how to do the 
following tasks? 


(Please mark one choice in each row) 


Toa Toa Toa ot at 
large moderate small all 
extent extent extent 
a) To display information in different ways IS2G29A 
b) To break a complex process into smaller parts IS2G29B 
C) To understand diagrams that describe or show IS2G29C 
real-world problems 
d) To plan tasks by setting out the steps needed to IS2G29D 
complete them 
e) To use tools to make diagrams that help solve IS2G29E 
problems 
f) To use simulations to help understand or solve real IS2G29F 
world problems 
g) To make flow diagrams to show the different parts IS2G29G 
of a process 
h) To record and evaluate data to understand and IS2G29H 
solve a problem 
i) To use real-world data to review and revise IS2G29l 


solutions to problems 


Q30 Do you study [computing, computer science, information technology, informatics or similar] in the 
current school year? 


IS2G30 


Yes 


No 
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Section 5: National contexts survey 


# 


IEA International Computer and 
Information Literacy Study 


National Contexts Survey 


May 2018 


Confidential to ICILS 2018 
Do not cite or quote 
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Introduction 


This survey questionnaire is addressed to National Research Coordinators (NRCs), who are 
asked to supply information about their country’s approach to developing computer information 
literacy among students. This will help provide important background information for interpretation 
of the data collected in other parts of the International Computer Information Literacy Study 
(ICILS). Your responses are vital in helping to provide a context for, and a better understanding 
of, the study results. 


Instructions 


We ask that you complete this survey questionnaire, working with others in your country as 
necessary (e.g., ministries and departments of education, relevant non-government organizations, 
specialist organizations concerned with supporting the application of educational technologies, 
and teachers associations). 


It is important that you answer each question carefully and concisely and provide additional 
information where requested so that an accurate picture of your country’s approach to computer 
information literacy is presented, particularly in relation to the target population (typically Grade 8). 


In order to help you to complete the survey as accurately and concisely as possible there are 
accompanying Notes for Guidance. It is recommended that you read these first before beginning 
to complete the survey and refer to them when answering the questions in the sections of the 
survey. 


Section C contains questions about ICT and learning at the lower secondary level (ISCED 2). In 
some countries, lower secondary education is taught as the second phase of primary or basic 
education. In this case, the questions should be answered with respect to the grades pertaining 
to this second phase of primary/basic education. Please refer to the UNESCO ISCED-11 
classification to identify the corresponding study programme in your country or education system: 


http://uis. unesco.org/sites/default/files/documents/international-standard-classification-of- 
education-isced-201 1-en.pdf 


Please complete this questionnaire at the latest by XXXXXXXX. Once you finished the 
questionnaire, please inform the ICILS International Study Center at ACER by sending a brief 
confirmation email to icils@acer.edu.au. 


ANALYZING THE ICILS 2018 DATA USING THE IEA IDB ANALYZER 


Section A. The education system 
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Who has responsibility for establishing the overarching goals and direction (e.g. global funding 
systems, curriculum development, establishment of assessment systems) for school education in 


your country? 


(Please mark one choice) 


Responsibility rests primarily with national ministry or department of education (1) 
Responsibility rests primarily with state/provincial jurisdictions (2) 
Responsibility rests primarily with local/municipal jurisdictions (3) 


Responsibility is shared between/across authorities at different levels — 
please describe below (4) 


Other 


Comments: 
Please elaborate your answer to Question 1 by describing the responsibility at 


national and state/provincial level for the provision of school education in your country. 


For what ages is school education compulsory in your country? 


(Please write a number in each box) 


NC2GA01 


NC2GA01T1 


NC2GA01T2 


At what age does compulsory education begin? NC2GA02A 


How many years of compulsory education are there? NC2GA02B 
Comments: 


Please elaborate your answer to Question 2 if there is more than one possible answer 
to either part of this question (i.e. if the ages and numbers of years can vary across 
the country). 


NC2GA02T 
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(b) 


(c) 
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Please outline the main characteristics of the institutions in which the following phases/cycles of 
education are provided in your country 


(a) Education at the primary level ISCED 1 NC2GA03AT 


You may need to differentiate and explain different types of school that operate. It would also be helpful 
to indicate the ways in which education for students with special needs is provided (e.g. mainly in special 
schools, in special classes within comprehensive schools or through an integrated system).If there are 
institutions that provide both ISCED 1 and ISCED 2 levels of education (as happens in some countries), 
please report on both levels for this institution under this question. 


Education at the lower secondary level <ISCED 2> NC2GA03BT 


You may need to differentiate and explain different strands, tracks or programmes that exist at institutional 
level. This could include strands, tracks or programmes concerning: 

e General education 

e Pre-vocational/pre-technical education 

e Vocational or technical education 


Education at the upper secondary level <ISCED 3> NC2GA03CT 


You may need to differentiate and explain different strands, tracks or programmes that exist at institutional 
level. This could include strands, tracks or programmes concerning: 

e General education 

e Pre-vocational/pre-technical education 

e Vocational or technical education 
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(d) 


The institutions in which education of the target grade mainly takes place. 


145 


NC2GA03DT 


In this response please indicate whether the target grade is most often located in an institution that provides 
primary and lower secondary education, lower secondary education only, or lower secondary and upper 
secondary education. It would be helpful to indicate the lowest grade and the highest grade included in 
the institutions that most often include the target grade and whether those institutions are comprehensive 


or specialized. 
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What is the approximate percentage of government (public) and non-government (private) schools 


that provide education at each ISCED level in your country? 


(Please write a percentage in each box and ensure that the percentages add to 100 in each row.) 


Public/government schools Private schools — Other schools (please describe): 


a. <ISCED 1> NC2GA04A01 NC2GA04A02 NC2GA04A03 
b. <ISCED 2> NC2GA04B01 NC2GA04B02 NC2GA04B03 
c. <ISCED 3> NC2GA04C01 NC2GA04C02 NC2GA04C03 


Please use this space to elaborate your response. 


NC2GA04T 


On the basis of the most recent data, what is the percentage of target grade students in: 


(Please write a percentage in each box and ensure that the percentages add to 100.) 


Public/government schools NC2GA05A 
Private schools NC2GA05B 
Other schools NC2GA05C 


Please use this space to elaborate your response and indicate the year in which 
that most recent data were collected. 


NC2GA05T 
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6a. | How much autonomy do schools with students in the target grade have regarding the selection and 
purchase of ICT equipment? 


(Please mark one choice per school type) 


School type 


Complete or a high level 
of autonomy 


Schools have full or almost full 
autonomy regarding how they 
operate in order to meet their 
statutory obligations. (1) 


Some autonomy 


Schools have some 
autonomy (including 
autonomy in some areas but 
not others) regarding how 
they operate in order to meet 
their statutory obligations. (2) 


Little or no autonomy 


Schools have little or no 
autonomy regarding how 
they operate in order to meet 
their statutory obligations. 
The authority (or authorities) 
with responsibility for school 
education control and specify 
xhow schools are to be run in 
order to meet their statutory 
obligations. (3) 


a. Public/government 
schools 
NC2GA06AA 


b. Private schools 
NC2GA06AB 


c. Other schools 
NC2GA06AC 


Please provide written comments to support your response, if necessary 


NC2GAO6AT 
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6b. | How much autonomy do schools with students in the target grade have regarding the selection and 
purchase of ICT equipment? 


(Please mark one choice per school type) 


School type 


Complete or a high level 
of autonomy 


Schools have full or almost 
full autonomy regarding the 
selection and purchase of ICT 
equipment. (1) 


Some autonomy 


Schools have some 
autonomy regarding the 
selection and purchase of ICT 
equipment (with, for example, 
the relevant education 
authority(ies) mandating the 
provision/purchase of some 
equipment). (2) 


Little or no autonomy 


Schools have little or no 
autonomy regarding the 
selection and purchase of ICT 
equipment. The authority (or 
authorities) with responsibility 
for school education control 
and specify schools’ 

selection and purchase of ICT 
equipment. (3) 


a. Public/government 
schools 
NC2GA06BA 


b. Private schools 
NC2GA06BB 


c. Other schools 
NC2GA06BC 


Please provide written comments to support your response, if necessary 


NC2GAO06BT 
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6c. How much autonomy do schools with students in the target grade have regarding the selection and 


purchase software? 


(Please mark one choice per school type) 


School type 


Complete or a high level 
of autonomy 


Schools have full or almost 
full autonomy regarding the 
selection and purchase of 
software. (1) 


Some autonomy 


Schools have some 
autonomy regarding the 
selection and purchase of 
software (with, for example, 
the relevant education 
authority(ies) mandating the 
provision/purchase of some 
software). (2) 


Little or no autonomy 


Schools have little or no 
autonomy regarding the 
selection and purchase of 
software. The authority (or 
authorities) with responsibility 
for school education control 
and specify schools’ selection 
and purchase of software. (3) 


a. Public/government 
schools 


NC2GA06CA 


b. Private schools 
NC2GA06CB 


c. Other schools 
NC2GA06CC 


Please provide written comments to support your response, if necessary 


NC2GA06CT 
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6d. How much autonomy do schools with students in the target grade have regarding staff participation in 
professional learning in the use of ICT? 


(Please mark one choice per school type) 


School type 


Complete or a high level 
of autonomy 


Schools have full or almost 
full autonomy regarding staff 
participation in professional 
learning in the use of ICT. (1) 


Some autonomy 


Schools have some 
autonomy regarding staff 
participation in professional 
learning in the use of 

ICT (with, for example, 

the relevant education 
authority(ies) mandating 
some Staff participation in 
professional learning). (2) 


Little or no autonomy 


Schools have little or no 
autonomy regarding staff 
participation in professional 
learning in the use of ICT. 

The authority (or authorities) 
with responsibility for school 
education control and specify 
all staff participation in 
professional learning in the 
use of ICT. (3) 


a. Public/government 
schools 


NC2GA06DA 


b. Private schools 
NC2GA06DB 


c. Other schools 
NC2GA06DC 


Please provide written comments to support your response, if necessary 


NC2GA06DT 
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6e. | How much autonomy do schools with students in the target grade have regarding ICT curriculum 


delivery?T 


(Please mark one choice per school type) 


School type 


Complete or a high level 
of autonomy 


Schools have full or almost 
full autonomy regarding ICT 
curriculum delivery. (1) 


Some autonomy 


Schools have some 
autonomy regarding ICT 
curriculum delivery (with, 
for example, the relevant 
education authority(ies) 
mandating some ICT 
curriculum delivery). (2) 


Little or no autonomy 


Schools have little or no 
autonomy regarding ICT 
curriculum delivery. The 
authority (or authorities) 

with responsibility for school 
education control and specify 
ICT curriculum delivery. (3) 


a. Public/government 
schools 


NC2GAO06EA 


b. Private schools 
NC2GAO6EB 


c. Other schools 
NC2GA06EC 


Please provide written comments to support your response, if necessary 


NC2GAO6ET 
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6f. How much autonomy do schools with students in the target grade have regarding the selection and 
appointment of their teachers? 


(Please mark one choice per school type) 


School type 


Complete or a high level 
of autonomy 


Schools have full or almost 
full autonomy regarding the 
selection and appointment of 
their teachers. (1) 


Some autonomy 


Schools have some 
autonomy regarding the 
selection and appointment of 
their teachers but the relevant 
education authority(ies) also 
has some control of the 
selection and appointment of 
teachers in schools. (2) 


Little or no autonomy 


Schools have little or no 
autonomy in the selection 
and appointment of their 
teachers. The authority (or 
authorities) with responsibility 
for school education control 
and specify the selection and 
appointment of teachers in 
schools. (3) 


a. Public/government 
schools 
NC2GAO06FA 


b. Private schools 
NC2GA06FB 


c. Other schools 
NC2GA06FC 


Please provide written comments to support your response, if necessary 


NC2GAO6FT 
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6g. How much autonomy do schools with students in the target grade have regarding the as-sessment of 
student achievement in computer and information literacy (or its equivalent)? 


(Please mark one choice per school type) 


School type 


Complete or a high level 
of autonomy 


Schools are free or largely 
free to conduct their own 
assessments of student CIL 
(or equivalent). (1) 


Some autonomy 


Schools have some 
autonomy in assessing 
student CIL, but this is 
combined with mandatory 
assessments from relevant 
school authorities. (2) 


Little or no autonomy 


Schools have little or no 
autonomy in the assessment 
of CIL. All or almost all 
student assessment of CIL 
is developed and initiated 

by the relevant education 
authority(ies). (3) 


a. Public/government 
schools 
NC2GA06GA 


b. Private schools 
NC2GA06GB 


c. Other schools 
NC2GA06GC 


Please provide written comments to support your response, if necessary 


NC2GA06GT 
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6h. | How much autonomy do schools with students in the target grade have regarding the provision of 
technical support for ICT? 


(Please mark one choice per school type) 


School type 


Complete or a high level 
of autonomy 


Schools have full or almost 
full autonomy regarding the 
provision of technical support 
for ICT. (1) 


Some autonomy 


Schools have some 
autonomy in enlisting and 
utilizing ICT technical support. 
This is combined with 
mandatory use of support 
from the relevant education 
authority(ies). (2, 


Little or no autonomy 


Schools have little or no 
autonomy regarding the 
provision of technical support 
for ICT. Technical support 

for ICT is controlled and 
managed by the relevant 
education authority(ies). (3) 


a. Public/government 
schools 
NC2GAQ06HA 


b. Private schools 
NC2GA06HB 


c. Other schools 
NC2GA0O6HC 


Please provide written comments to support your response, if necessary 


NC2GAO6HT 
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Section B. Plans and policies for using ICT in education 


The questions in Section B relate to general policies and overall approaches to ICT in school education in 
your country rather than specifically to the target grade. Questions that address the approaches to the use 
of ICT in the target grade will be asked later in the questionnaire. 


Ts Are there plans or policies supporting the use of ICT in school education? 


(Please mark one choice) 


Yes, at the national level NC2GB07A 
Yes, at the state/provincial level NC2GB07B If yes, please continue with 
a = question 8 
Yes, at the local/district/municipal level NC2GB07C 
No, neither at the national state/provincial NC2GB07D If no, please go to 
level or local/district/municipal level question 15 
Comments: 
Please use this space to elaborate your response. NC2GB07T 
8. What are the key documents that outline the plans and policies for supporting the use of ICT for 


teaching and learning purposes in school education in your country 


(Please provide a URL for each document if possible) NC2GBO8T 
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9. Have there been any major changes introduced relevant to the approach and use of ICT in education 
in your country in the last 7 years? 


(Please mark one choice) 


NC2GB09 


Yes (1) 


No (2) 


Comments: 
If you answered ‘yes’, please describe, in brief, the nature of the changes and provide 


links/references to relevant documents: NC2GBO9T 
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10. To what extent do the plans or policies for using ICT in education emphasize improving student 
learning with specific mention of the following aspects: 


(Please mark one choice on each row) 


They explicitly They imply that They place no 


state this this aspect is — emphasis on this 
aspect as an important, but it aspect (3) 
intended focus or is not explicitly 
outcome (1) stated as an 


intended focus 
or outcome (2) 


a) Subject matter content (Language Arts, NC2GB10A 
Science, etc.) 

b) Preparing students for using ICT in their NC2GB10B 
future work 

c) Developing information literacy NC2GB10C 

d) ICT-based skills in critical thinking, NC2GB10D 


collaboration and communication 


e) Increasing access to online courses of NC2GB10E 
study (e.g. for rural students) 


f) Computer programming or developing NC2GB10F 
applications for digital devices 


g) Responsible and ethical use of digital NC2GB10G 
devices including cyber-safety 


Comments: 
Please use this space to elaborate your response. NC2GB10T 
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11. | To what extent do the plans or policies for using ICT in education emphasize the importance of the 
following resources? 


(Please mark one choice on each row) 


They explicitly They imply They place no 
state the need for the need for emphasis on this 
the resource (1) the resource resource (3) 
without 
explicitly 
stating that it is 
required (2) 

a) Provision of computer equipment and NC2GB11A 

other ICT resources 
b) Maintenance of computer equipment NC2GB11B 

and other ICT resources 
c) Renewal, updating and replacement NC2GB11C 

of computer equipment and other ICT 

resources 
d) Support for teachers for using computer NC2GB11D 

equipment and other ICT resources in 

their work 
e) Access to digital educational resources NC2GB11E 
f) Internet connectivity NC2GB11F 
g) Home access to school-based digital NC2GB11G 


education resources such as through 
school-hosted online portals 


Comments: 
Please use this space to elaborate your response. NC2GB11T 
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12. To what extent do the plans or policies for using ICT in education emphasize the following methods 
of supporting student learning? 


(Please mark one choice on each row) 


They explicitly They imply the They place_no 
state the need for need for this emphasis on this 
this method (1) method without method (3) 
explicitly stating 
it (2) 


a) Pre-service teacher education in the NC2GB12A 
use of ICT 

b) In-service teacher education in the use NC2GB12B 
of ICT 

c) The use of learning management NC2GB12C 
systems 

d) Reporting to parents NC2GB12D 

e) Providing feedback to students NC2GB12E 

Comments: 


Please use this space to elaborate your response. NC2GB12T 
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13. To what extent do the plans and policies for the use of ICT in education include the following aspects 
as priorities? 
(Please mark one choice on each row) 


Not indicated as The need for The aspect is 
a priority (1) this aspect explicitly stated 
Is implicit in in the plans and 
the plans and policies (3) 


policies without 
being explicitly 


stated (2) 

a) Professional development for teachers’ NC2GB13A 
pedagogical use of ICT 

b) Sufficient ICT infrastructure and NC2GB13B 
resources in schools 

c) Development of ICT-related NC2GB13C 
competencies in students 

d) Development and provision of digital NC2GB13D 
learning materials 

e) Reduction of the digital divide between NC2GB13E 
groups of students 

f) Improvement of administrative and NC2GB13F 
management systems in schools 

g) Use of ICT to improve communication NC2GB13G 


with parents 


Please provide any further comments to support your responses above and also detail any other aspects of 
ICT in education that are indicated as priorities in plans and policies. 


NC2GB13T 
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14. Do the plans and/or policies for using ICT in education refer to providing 1:1 computing in schools? 


(Please mark one choice) 


NC2GB14 
Yes, plans and/or policies for ICT in education refer to providing 1:1 computing in schools (1) 
No, plans and/or policies for ICT in education do not refer to providing 1:1 computing in 
schools (2) 
Please indicate the targets for computer provision in schools or indicate NC2GB14T 


that there are no targets. 


15. Is there formal support for the development of digital learning resources (e.g. digital curriculum 
resources or learning objects) through government agencies, incentives for other agencies, or 
encouragement to publishers to produce these resources? 


(Please mark one choice) 


NC2GB15 
Yes, there is formal support for the development of digital resources (1) 
No, there is no formal support for the development of digital resources (2) 
Comments (please use this space to describe any support for the NC2GB15T 


development of digital resources): 
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16a. How is CIL education intended to be taught at the level of primary education (ISCED Level 1) 
(Please mark one choice on each row) 


Yes, as a Yes, as anon- No (3) 
compulsory compulsory 
subject (1) subject (2) 


It is taught as a separate subject by NC2GB16AA 
specializing in ClL-related education (such 
as ICT literacy/digital literacy/computing) 


If you have selected yes, 
a) please list the name of the subject(s) with an English translation (if applicable) 


b) indicate whether the subject includes “coding” as well as applications. NC2GB16AAT 


It is integrated into Science and Technology NC2GB16AB 
studies deemed to be relevant to CIL 


It is integrated into a range of other subjects NC2GB16AC 
taught at school 


Comments: 
Please use this space to elaborate your response. 


NC2GB16AT 
16b. How is CIL education intended to be taught at the level of lower secondary education 
(ISCED Level 2 — the ICILS target grade) 
(Please mark one choice on each row) 
yes, as a Yes, as a non- No (3) 
compulsory compulsory 
subject (1) subject (2) 


It is taught as a separate subject by teachers NC2GB16BA 
specializing in ClL-related education (such as 
as ICT literacy/digital literacy/computing) 


If you have selected yes, 
a) please list the name of the subject(s) with an English translation (if applicable) 


b) indicate whether the subject includes “coding” as well as applications. 


APPENDIX A 


It is integrated into Science and Technology NC2GB16BB 


studies deemed to be relevant to CIL 


It is integrated into a range of other subjects NC2GB16BC 


taught at school 
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Comments: 
Please use this space to elaborate your response. 


NC2GB16BT 
16c. How is CIL education intended to be taught at the level of upper secondary education 
(ISCED Level 3) 
(Please mark one choice on each row) 
Yes, as a Yes, as a non- No (3) 
compulsory compulsory 
subject (1) subject (2) 
It is taught as a separate subject by NC2GB16CA 
teachers specializing in ClL-related 
education (such as ICT literacy/digital 
literacy/computing) 
If you have selected yes, 
a) please list the name of the subject(s) with an English translation (if applicable) 
b) indicate whether the subject includes “coding” as well as applications. NC2GB16CAT 


It is integrated into Science and Technology NC2GB16CB 
studies deemed to be relevant to CIL 


It is integrated into a range of other subjects NC2GB16CC 
taught at school 


Comments: 
Please use this space to elaborate your response. 


NC2GB16CT 
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17. Is there mandated assessment of ICT and computing related skills of students in the target grade? ) 


(Please mark one choice) NC2GB17 


Yes, using a compulsory assessment for all students at the national and/or state/provincial level (1) 


Yes, using a sample-based assessment at the national and/or state/provincial level (2) 


Yes, using a non-compulsory common assessment (3) 


Yes, but assessment is controlled at the school level (4) 


There is no mandated requirement for assessing students in this area (5) 


Comments: 
Please use this space to elaborate your response. 


NC2GB17T 
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Section C. ICT and student learning at lower secondary level (ISCED 2) 


In responding to the questions in this section please use the response categories to indicate your general 
answer for lower secondary education (ISCED 2). In addition please use the comment boxes to indicate: 


* whether the national and/or state/provincial education authorities regard the issue as a priority; 


* whether there are any projects/programs that promote these practices (and, if possible, briefly describe 
these projects or programs); and 


* whether national and/or state or provincial education authorities collect data on the extent of the 
practice. 


18. In the past seven years have education authorities (at any of the national, state/provincial or local/ 
municipal level) used, or supported the use of, ICT for the provision of the following types of student 
assessment? 


(Please mark one choice on each row) 


Yes No 
(1) (2) 


a) Diagnostic assessments | NC2GC18A | 
a) Formative assessments NC2GC18B 
b) Summative assessments NC2GC18C 
c) National or state/provincial monitoring programs NC2GC18D 


Please give examples for each assessment type for which you have selected ‘Yes’: 


NC2GC18T 
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9. To what extent does the national curriculum emphasize the following aspects of CIL? 


(Please mark one choice on each row) 


This aspect is The value of The curriculum 


explicitly stated in this aspect is places no 
the curriculum (1) implied without | emphasis on this 
being explicitly aspect (3) 
stated (2) 
a) Searching for information using ICT NC2GC19A 
b) Evaluating the reliability of information NC2GC19B 
sources accessed using the Internet 
c) Presenting information for a given NC2GC19C 
audience or purpose using ICT 
d) Organizing information obtained from NC2GC19D 
Internet sources 
e) Issues relating to intellectual property such NC2GC19E 
as copyright and attribution sources 
f) Responsible and respectful publication NC2GC19F 
of information 
g) Use of productivity tools (Such as word NC2GC19G 
processing, spreadsheet and 
presentation software) 
h) IT security issues (e.g. passwords, NC2GC19H 
malware, phishing) 
i) Data security (Such as the collection of NC2GC191 
Internet use data by search engines 
and social media sites) 
Please use this space to elaborate your response. NC2GB19T 
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20. To what extent does the national curriculum_emphasize the following aspects of CIL? 
(Please mark one choice on each row) 


This aspect is The value of The curriculum 
explicitly stated in this aspect is places no 
the curriculum (1) implied without | emphasis on this 

being explicitly aspect (3) 
stated (2) 
a) Planning technology-based products or NC2GC20A 
solutions 
b) Developing technology-based products NC2GC20B 


or solutions to meet user requirements 


c) Evaluating and refining technology-based NC2GC20C 
products or solutions 


d) Creating visual representations NC2GC20D 
(e.g. flow charts and decision trees) of 
processes 


e) Creating visual representations NC2GC20E 
(e.g. tables, graphs or charts) of 
information/data 


f) Designing user interfaces for technology- NC2GC20F 
based products or solutions 


g) Revising technology-based products or NC2GC20G 
solutions on the basis of user feedback 
or other data 


h) Creating algorithms NC2GC20H 

i) Writing code, programs or macros NC2GC20! 

j) Evaluating code, programs or macros NC2GC20J 

k) Developing digital applications NC2GC20K 
(e.g. programs/apps) 

) Identifying and describing the properties NC2GC20L 


of digital systems 


Please use this space to elaborate your response. NC2GB20T 
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Section D. ICT and teacher development 


Please indicate the extent to which there is provision for teacher development related to the use of ICT. 


In responding to the questions in this section please use the response categories to indicate your general 


answer. In addition please use the comment boxes to indicate: 


* whether national and/or state/provincial education authorities regard the issue as a priority; and 


* whether there are any initiatives, projects, or programs in which ICT is used to support teacher 
development (and, if possible, briefly describe these initiatives, projects or programs). 


21. | How do education authorities (at the national, state/provincial or local/municipal level) require the 
development of teacher capacity to use ICT with regard to the following areas? 


(Please mark as many choices as apply in each row) 


This is a Eligibility for Participation This is not 
mandatory registration as a in some form required 
component teacher requires of professional 

of pre-service evidence of this learning 
teacher capacity program in this 
education area is required 
of teachers 
a) Technical capacity in using ICT 
NC2GC21AA NC2GC21AB NC2GC21AC NC2GC21AD 
b) Using ICT in pedagogy 
NC2GC21BA NC2GC21BB NC2GC21BC NC2GC21BD 
c) Collaboration and communication 
using ICT 
NC2GC21CA NC2GC21CB NC2GC21CC NC2GC21CD 
d) Using ICT for student assessment 
NC2GC21DA NC2GC21DB NC2GC21DC NC2GC21DD 
Please describe the ways in which teacher development in these NC2GD21T 


areas is provided or required. 
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22. | How do education authorities (at the national, state/provincial or local level) support teacher access 
to, and participation in, ICT-focused professional learning programs for the following purposes? 


(Please mark as many choices as apply in each row) 


By funding By providing By providing Not at all 
teacher resources for relieving 
participation in teachers to teachers to 
programs access allow regular 
teachers 
to attend 
programs 


a) To improve ICT/technical skills 


NC2GC22AA NC2GC22AB NC2GC22AC NC2GC22AD 


b) To improve content knowledge with 
respect to CIL 


NC2GC22BA NC2GC22BB NC2GC22BC NC2GC22BD 


c) To improve teaching skills with respect to 
ClL-related content 


NC2GC22CA NC2GC22CB NC2GC22CC | | NC2GC22CD 


d) To develop digital teaching and learning 
resources 


NC2GC22DA NC2GC22DB NC2GC22DC | | NC2GC22DD 


e) To integrate ICT in teaching and learning 
activities 


NC2GC22EA NC2GC22EB NC2GC22EC NC2GC22ED 


f) To improve skills in computer 
programming or developing applications 
for digital devices 


NC2GC22FA NC2GC22FB NC2GC22FC NC2GC22FD 


Please comment with examples of programs: NC2GD22T 
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Section F. ICT-based learning and administrative management systems 


Please indicate the extent to which ICT is used to support learning and administrative management systems 
with respect to any learning area (i.e. not limited to ICT literacy, computing or related areas). 


In responding to the questions in this section please use the response categories to indicate your general 
answer. In addition, please use the comment boxes to indicate: 


* whether the national and/or state/provincial education authorities regard the issue as a priority; and 


* whether there are any projects or programs that promote these practices (and, if possible, briefly 
describe these projects or programs). 


23. Do ministries or departments of education at national and/or state/provincial level use I|CT-based 
data systems for the following purposes? 


(Please mark one choice on each row) 


Yes No 
(1) (2) 


a) Collecting, analyzing and reporting student NC2GC23A 
achievement data at various levels of aggregation 

b) Providing links to examples of student work and NC2GC23B 
teaching resources that are related to achievement 
data 

c) Providing tools to schools for analysis of data about NC2GC23C 


the school and its environment 


d) Providing tools to schools for analysis of data about NC2GC23D 
the school and its environment 


Please give examples: 


NC2GC23T 
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24. Doministries or departments of education at national and/or state/provincial level pro-vide training 
for teachers in the use of ICT for the following activities? 


(Please mark one choice on each row) 


Yes No 
(1) (2) 
a) Interpreting data at school, class and/or student level NC2GC24A 
b) Linking data to instructional decisions NC2GC24B 
c) Using data to monitor student progress over time NC2GC24C 
Please give examples: 
NC2GC24T 


25. Please list any sources of information that you used in addition to those you have listed in response 
to question 8 to answer this questionnaire 


Sources: 


NC2GC25T 


THANK YOU 


for taking the time to complete this survey. 
Your response is very important to us. 
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APPENDIX B: 


National adaptations of international 
questionnaires 


Overview 


This appendix describes national adaptations made to the international version of the ICILS 
2018 questionnaires. This information provides users with a guide to evaluate the availability 
of internationally comparable data for use in secondary analyses involving the ICILS 2018 
questionnaire variables. 


Questionnaire adaptations include questions that countries were required to adapt, questions 
that were not administered, and questions that countries modified to suit their national context. 


The adaptations to questionnaires are presented in five sections: 
e General adaptations; 


e Adaptations in student questionnaire; 


e Adaptations in principal questionnaire; 
e Adaptations in ICT coordinator questionnaire; and 


e Adaptations in teacher questionnaire. 


List of general adaptations across questionnaires 
Adaptations of <Target grade> 


Country Adaptation 
CHL Grade 8 
DEU Grade 8 
DNK 8th grade 
FIN 8th grade 
FRA Fourth 
TA Grade 8 
AZ 8th grade 
OR Middle school 2nd grade 
LUX 6th/8th 
PRT 8th grade 
RMO Grade 8 
URY 2nd year 
USA Eighth-grade 
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Adaptations of <Target grade> 


Country Adaptation 
CHL Job 
DEU Profession 
DNK Job 
FIN Job 
FRA Job 
TA Job 
AZ Job 
OR Job 
LUX Job 
PRT Professional activity 
RMO Job 
URY Job 
USA Job 


Adaptations of <Guardian 1> 


Country Adaptation 
CHL om or Guardian 
DEU other or female guardian (e.g., stepmother or foster mother) 
DNK Parent or carer 1 
FIN Parent or guardian 1 
FRA Parent or person responsible 1 
TA Parent/Guardian A 
AZ other or a female guardian 
OR other or female guardian 
LUX Parent or guardian 1 
PRT Father or guardian 
RMO other or guardian 
URY other or guardian 1 
USA Parent or guardian 1 


174 


Adaptations of <Guardian 2> 
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Country Adaptation 

CHL Dad or Guardian 
DEU Father or male guardian (e.g., stepfather or foster father) 
DNK Parent or carer 2 
FIN Parent or guardian 2 
FRA Parent or person responsible 2 
TA Parent/Guardian B 

AZ Father or amale guardian 

OR Father or male guardian 
LUX Parent or guardian 2 
PRT other or guardian 
RMO Father or guardian 
URY Father or guardian 2 
USA Parent or guardian 2 
Adaptations of <Smartphones> 
Country Adaptation 

CHL Smartphones 

DEU Smartphones 

DNK Smartphones 

FIN Smartphones 

FRA Smartphones 

TA Smartphones 

AZ Smartphones 

OR Smartphones 

LUX Smartphones 

PRT Smartphones 

RMO Smartphones 

URY Smartphones 

USA Smartphones 
Adaptations of <App> 

Country Adaptation 

CHL Application 

DEU App 

DNK App 

FIN Application 

FRA Application 

TA App 

AZ Application 

OR App 

LUX App 

PRT App 

RMO App 

URY App 

USA App 
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Adaptations of <School definition> 


Country Adaptation 
CHL A school is a unit with a defined number of teachers and students, it can include 
different programs or specialties, and it corresponds to a single National ID 
DEU Omitted 
DNK A school is one whole unit with a defined number of teachers and students, which can 
include different programs or tracks. The definition of ‘school’ should be based on the 
environment that is shared by students, which is usually a shared faculty, set of buildings, 
social space and also often includes a shared administration and charter 
FIN The school in this survey is the entity that operates in the same premises, under the 
leadership of the same principal, and to take advantage of shared resources. The school 
can therefore be either a mere secondary school or a broader comprehensive school 
FRA Your institution as a whole 
TA Inthe questionnaire, the term ‘school refers to the building sampled inside of the ‘institute’ 
AZ The school is an educational institution that implements the general education curricula 
of primary, basic secondary, and general secondary education 
OR Omitted 
LUX One whole unit with a defined number of teachers and students, which can include 
different programs or tracks. The definition of ‘school’ should be based on the environment 
that is shared by students, which is usually a shared faculty, set of buildings, social space 
and also often includes a shared administration and charter 
PRT The school attended by the 8th grade students who were selected to participate in ICILS. 
That is to say, if the organic unit of school that you manage is a cluster of schools, you must 
consider the specific characteristics of the specific school that is attended by the selected 
students, i.e., the number of students, the teachers, the resources, and educational offer 
of that particular establishment 
RMO An educational organization providing training in general education 
URY The center that you direct 
USA Omitted 


Adaptations of <School of the sampled students> 


Country Adaptation 
CHL Your school 
DEU Your school as a whole 
DNK The school which the sampled students are enrolled at 
FIN The school which has been selected for this study as a whole 
FRA All secondary school classes 
TA Omitted 
AZ Your school as a whole 
OR Your school 
LUX The school of the interviewed students 
PRT The school of the sampled students 
RMO The school of the sampled students 
URY The school of the sampled students 
USA Your school as a whole 
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APPENDIX C: 


Variables derived from the questionnaire 
data 


Overview 


This appendix contains documentation on all the derived variables contained in the ICILS 2018 
data files that are based on questionnaire variables. These variables were used to report data 
n the ICILS 2018 international reports, and they have been made available as part of the ICILS 
2018 international database (IDB) so that researchers can use them in secondary analyses. 
This appendix has four sections corresponding to each questionnaire from which the reporting 


variables are derived: 


e Section 1: Student questionnaire 


e Section 2: Principal questionnaire 


e Section 3: ICT coordinator questionnaire 


e Section 4: Teacher questionnaire 


Each section lists first the simple indices and then the scale indices, as derived from survey 
variables, in the order of the variables that were used to derive the variable as they appear in the 


instruments, respectively. 


The following information is provided for each derived variable: 


Variable name: The name of the derived variable 


Description: A description of the variable content 
Procedure: A procedural description of how the derived variable was computed 
Source: Source variables used to derive scale or index 


Interpretation: A description how to interpret the scale 
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Section 1: Student questionnaire 


Simple indices 


Variable Name: S_AGE 
Description: Age of student 
Procedure (T,,- S,,) 
SAGE = (T,-5,) +5 
where T, and S, are, respectively, the year of the test and the year of birth of the tested student, in four-digit 
format (e.g., “2018” or “2003"), and where T, and S_ are, respectively, the month of the test and the month of 
the student's birth. 
Source: When were you born? 
Month IS2GO1A 
Year IS2GO01B 
Variable Name: S SEX 
Description: Sex of student 
Procedure: Simple recoding 
Source: Are you a girl or a boy? IS2GO2 Recoding 
1=Girl 1 1 
2 = Boy Z O 
Variable Name: S_ISCED 
Description: Expected ISCED by student 
Procedure: Simple recoding 
Source: What is the highest levels of education you expect to complete? 
(Please mark only one choice) IS2G03 Recoding 
1=[ISCED level 6, 7 or 8] 1 4 
2 =[ISCED level 4 or 5] 2 3 
3 = [ISCED level 3] 3 2 
4=[ISCED level 2] 4 1 
5 =| donot expect to complete [ISCED level 2] 5 O 
Variable Name: S_IMMIG 
Description: mmigration status 
Procedure: Simple recoding 
Source: n what country were you and your parents born? 
Please mark only one choice in each column) 
You IS2G04A 
Parent or guardian 1] |S2G04B 
Parent or guardian 2] |S2G04C 


O = Students and/or at least one parent born in country of test 
1 = Student born in country of test but both/only parent(s) born abroad 
2 = Student and both/only parent(s) born abroad 
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Variable Name: S_IMMBGR 


Description: Immigration status (dummy coded) 
Procedure: Simple recoding 
Source: O = Students and/or at least one parent born in country of test O O 
1 = Student born in country of test but both/only parent(s) bornabroad = 1. 1 
2 = Student and both/only parent(s) born abroad Z al 
Variable Name: S TLANG 
Description: Test language spoken at home 
Procedure: Simple recoding 
Source: What is the highest levels of education you expect to complete? IS2GO05 Recoding 
(Please mark only one choice) 
1 = [Language of test] 1 1 
2 = [Other language 1] 2 O 
3 = [Other language 2] 3 O 
4 = [Another language] 4 O 
Variable Name: S P1WORK 
Description: Paid work status of parent 41 
Procedure: Simple recoding 
Source: Does your [parent or guardian 1] work in a paid [job]? |S2G06 Recoding 
Yes 1 1 
No 2 O 
Variable Name: S P1ISCO 
Description: ISCO of parent 1 
Procedure: The occupation codes are based on the ISCO-08 framework. 
Source: What is your [parent or guardian 1]’s main [job]? |S2GO07A 
for example high school teacher, kitchen-hand, sales manager) 
Please write in the [job] title) 
What was your [parent or guardian 1]’s last main [job]? |S2G07B 
for example high school teacher, kitchen-hand, sales manager) 
Please tell us his/her last main [job]. If he/she has never had a paid [job], 


please write what he/she is currently doing) 
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Variable Name: 


S_P1ISEl 


Description: ISEI of parent 1 
Procedure: Simple recoding 
Source: What does your [parent or guardian 1] do in his/her main [job]? IS2GO08A 


for example teaches high school students, helps the cook prepare 
meals in a restaurant, manages a sales team) 


Please use a sentence to describe the kind of work he/she does in that [job]) 


for example taught high school students, helped the cook prepare 
meals in a restaurant, managed a sales team) 


Please use a sentence to describe the kind of work he/she did in that [job] 
or what he/she is currently doing if he/she never had a paid [job]) 


What did your [parent or guardian 1] do in his/her last main [job]? IS2GO08B 


Variable Name: 


S_P1ISCED 


Description: ISCED of parent 1 

Procedure: Simple recoding 

Source: What is the highest level of education completed by IS2GO09 Recoding 
[parent or guardian 1]? 
If you are not sure which box to choose, please ask the [test administrator] for help. 
(Please mark only one choice) 
1=[ISCED level 6, 7, or 8] 41 4 
2 =[ISCED level 4 or 5] 2 3 
3 = [ISCED level 3] 3 2 
4 =[ISCED level 2] 4 1 
5 = He/she did not complete [ISCED level 2] 5 0 

Variable Name: S P2WORK 

Description: Paid work status of parent 2 

Procedure: Simple recoding 

Source: Does your [parent or guardian 2] work in a paid [job]? |S2G10 Recoding 
1=Yes 1 1 
2=No 2 O 

Variable Name: S P2ISCO 

Description: Occupation of the student’s father 

Procedure: The occupation codes are based on the ISCO-08 framework. 

Source: What is your [parent or guardian 2]’s main [job]? IS2G11A 
for example high school teacher, kitchen-hand, sales manager) 
Please write in the [job] title) 
What was your [parent or guardian 2]’s last main [job]? IS2G11B 
for example high school teacher, kitchen-hand, sales manager) 


Please tell us his/her last main [job]. If he/she has never had a paid [job], 
please write what he/she is currently doing) 


Please write in the [job] title) 
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Variable Name: 


S_P2ISEI 


Description: ISEI of parent 2 
Procedure: Simple recoding 
Source: What does your [parent or guardian 2] do in his/her main [job]? IS2G12A 


for example teaches high school students, helps the cook prepare 
meals in a restaurant, manages a sales team) 


Please use a sentence to describe the kind of work he/she does in that [job]) 


What did your [parent or guardian 2] do in his/her last main [job]? 1S2G12B 
for example taught high school students, helped the cook prepare 
meals in a restaurant, managed a sales team) 
Please use a sentence to describe the kind of work he/she did in that [job] 
or what he/she is currently doing if he/she never had a paid [job]) 
Variable Name: S P2ISCED 
Description: ISCED of parent 2 
Procedure: Simple recoding 
Source: What is the highest level of education completed by your IS2G13 Recoding 
parent or guardian 2]? 
If you are not sure which box to choose, please ask the [test administrator] for help. 
Please mark only one choice) 
ISCED level 6, 7, or 8] 4 4 
ISCED level 4 or 5] 2 3 
ISCED level 3] 3 2 
ISCED level 2] 4 1 
He/she did not complete [ISCED level 2] 5 0 


Variable Name: 


S_HISEI 


Description: Highest ISEI of parents 
Procedure: S_HISEl=max(S_P1ISEI,S_P2SE]) 
Source: S P1lSEI, S_P2SEI (see above) 


Variable Name: S_HISCED 
Description: Highest ISCED of parents 
Procedure: S HISCED=max(S_P1ISCED,S_P2ISCED) 
Source: S_P1ISCED,S_ P2ISCED (see above) 
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Variable Name: 


S_HOMLIT 


Description: 


Home literacy index 


Procedure: 


Simple recoding 


Source: 


About how many books are there in your home? 


(for example high school teacher, kitchen-hand, sales manager) 
your schoolbooks. 


(Please mark only one choice) 


1= None or very few (0-10 books) 

2 = Enough to fill one shelf (11-25 books) 

3 = Enough to fill one bookcase (26-100 books) 

4 = Enough to fill two bookcases (101-200 books) 


5 = Enough to fill three or more bookcases (more than 200 books) 


IS2G14 


Recoding 


BwWN PF OC 


Variable Name: 


S_INTNET 


Description: 


Internet access at home 


Procedure: 


Simple recoding 


Source: 


Do you have an Internet connection at home? 
(Please mark only one choice) 

1=Yes 

2=No 


IS2G15B 


Recoding 


Variable Name: 


S_EXCOMP 


Description: 


Computer experience in years (S EXCOMP) 


Procedure: 


Simple recoding 


Source: 


How long have you been using each of the following types of ICT devices? 


(Please mark only one choice) 

Desktop or [laptop] computers 

1 = Never or less than one year 

2 = At least one year but less than three years 
3 = At least three years but less than five years 
4 = At least five years but less than seven years 


5 = Seven years or more 


IS2G16A 


mam BR WwW NY 


Recoding 
O 


BR wON BF 


Variable Name: 


S_EXTAB 


Description: 


Tablet experience in years (S_EXTAB) 


Procedure: 


Simple recoding 


Source: 


Please mark only one choice) 

Tablet devices or e-readers (e.g., [iPad, Tablet PC, Kindle]) 
1 = Never or less than one year 

2 = At least one year but less than three years 

3 = At least three years but less than five years 

4 = At least five years but less than seven years 


5 = Seven years or more 


How long have you been using each of the following types of ICT devices? 


IS2G16B 


mam BR WN 


Recoding 
0 


BR WON FB 
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Variable Name: S_EXSMART 


Description: Smartphone experience in years (S_ EXSMART) 
Procedure: Simple recoding 
Source: How long have you been using each of the following types of ICT devices? 


(Please mark only one choice) 
[Smartphones] except for using text and calling IS2G16C Recoding 
1 = Never or less than one year 1 O 

2 = At least one year but less than three years 


3 = At least three years but less than five years 


4 = At least five years but less than seven years 


On BR WN 
BwWN PB 


5 = Seven years or more 


Variable Name: S_ICTSTUD 


Description: ICT studies in current school year 


Procedure: Simple recoding 


Source: Do you study [computing, computer science, information IS2G30 Recoding 
technology, informatics, or similar] in the current school year? 


(Please mark only one choice) 
1 = Yes 41 
2=No 2 O 
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Variable Name: 


S_NISB 


Description: National index of socio-economic background 
Procedure: Scale scores with mean of O and standard deviation of 1 for equally weighted countries 
Source: Derived from highest occupational status of parents (S_HISEI), highest educational level of parents 


(S_HISCED: collapsed the lowest two categories to have an indicator variable with four categories), 


and the number of books at home (S_HOMLIT: collapsing the two highest categories) (See above) 


Variable Name: 


S_GENACT 


Description: 


Use of general applications for activities 


Procedure: 


RT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 


Interpretation 


igher values indicating more frequent use 


Source: 


ow often do you use ICT for each of the following activities? 
Please mark one choice in each row) 


Write or edit documents IS2G19A 
Use aspreadsheet to do calculations, store data, or plot graphs IS2G19B 
e.g., using [Microsoft Excel®]) 
Create a simple “slideshow” presentation IS2G19C 
e.g., using [Microsoft Power Point®]) 
Recoding 
1 = Never ie) 
2 = Less than once a month 1 
3 = At least once a month but not every week 2 
4 = At least once a week but not every day 3 
5 = Every day 4 
Variable Name: S SPECACT 
Description: Use of specialist applications for activities 
Procedure: RT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 


Interpretation 


igher values indicating more frequent use 


Source: 


ow often do you use ICT for each of the following activities? 
Please mark one choice in each row) 


Record or edit videos $2G19D 
Write computer programs, scripts, or apps S2G19E 
e.g., using [Logo, LUA, or Scratch]) 
Use drawing, painting, or graphics software or [apps] S2G19F 
Produce or edit music $2G19G 
Build or edit a webpage $2G19H 
Recoding 
1 = Never 0 
2 = Less than once amonth 1 
3 = At least once a month but not every week 2 
4 = At least once a week but not every day 3 
5 = Every day 4 
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Variable Name: 


Variable Name: S USECOM 


Description: 


Use of ICT for social communication 


Procedure: 


RT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 


Interpretation 


igher values indicating more frequent use 


Source: ow often do you use ICT for each of the following communication activities? 
Please mark one choice in each row) 
Share news about current events on social media $2G20A 
Communicate with friends, family, or other people using instant S$2G20B 
messaging, voice or video chat (e.g. [Skype, WhatsApp, Viber]) 
Send texts or instant messages to friends, family, or other people S2G20C 
Write posts and updates about what happens in your life on S2G20D 
social media 
Post images or video in social networks or online communities $2G20H 
(e.g., [Facebook, Instagram or YouTube]) 
Watch videos or images that other people have posted online S$2G20 
Send or forward information about events or activities to $2G20) 
other people 
Recoding 
1 = Never 1 O 
2 = Less than once amonth 2 il 
3 = At least once a month but not every week 3 2 
4 = At least once a week but not every day 4 3 
5 = Every day 5 4 
Variable Name: S_USEINF 


Description: 


Use of ICT for exchanging information 


Procedure: 


RT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 


Interpretation 


igher values indicating more frequent use 


Source: 


ow often do you use ICT for each of the following communication activities? 
Please mark one choice in each row) 


Ask questions on forums or [Q&A, question and answer] websites IS2G20E 

Answer other peoples’ questions on forums or [Q&A, question IS2G20F 

and answer] websites 

Write posts for your own blog (e.g., [WordPress, Tumblr, Blogger]) IS2G20G 

Recoding 

1 = Never 1 O 

2 = Less than once amonth 2 1 

3 = At least once a month but not every week 3 2 

4 = At least once a week but not every day 4 3 

5 = Every day 5 4 
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Variable Name: S ACCONT 
Description: Use of ICT for accessing content from the internet 
Procedure: RT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 


Interpretation 


igher va 


ues indicatin 


gm 


ore frequent use 


Source: 


Searc 
activi 


Read 
Read 


Searc 


some 


ow often do you use 
Please mark one choice 


h the Internet to 
ies to do 


reviews on the In 


find 


Leh 


news stories ont 


hing 


1 = Never 


2 = Less than once a month 
3 = At least once a month but not every week 


4 = At least once a week but not every day 


5 = Every day 


h for online informa 


ne 


CT to do each of the following leisure activities? 
in each row) 


information about places to go or S$2G21A 


$2G21 
$2G21 


net of things you might want to buy 


nternet 


Use websites, forums, or online videos to find out how to do 


tion about things you are interested in 


$2G21 
$2G21 


™1™O 0 W 


Recoding 
0 


OB WDNR 
BWN PR 


Variable Name: 


S_USESTD 


Description: 


Use of ICT for study purposes 


Procedure: 


RT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 


Interpretation 


igher values indicating more frequent use 


Source: 


ow often do you use ICT for the following school-related purposes? 
Please mark one choice in each row) 


Prepare reports or essays S2G22A 
Prepare presentations $2G22B 
Work online with other students $2G22C 
Complete [worksheets] or exercises §2G22D 
Organize your time and work $2G22E 
Take tests $2G22F 
Use software or applications to learn skills or a subject S2G22G 
(e.g., mathematics tutoring software, language learning software) 
Use the Internet to do research $2G22H 
Use coding software to complete assignments (e.g., [Scratch]) S$2G22 
Make video or audio productions S$2G225) 

Recoding 
1 = Never 1 0 
2 = Less than once amonth 2 1 
3 = At least once a month but not every week 3 2 
4 = At least once a week but not every day 4 3 
5 = Every day 5 4 
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Variable Name: S_GENCLASS 


Description: Use of general applications in class 
Procedure: RT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 
Interpretation Higher values indicating more frequent use 
Source: When studying throughout this school year, how often did you use the following tools during class? 
Please mark one choice in each row) 
Word-processing software (e.g., [Microsoft Word®)]) |S2G24B 
Presentation software (e.g., [Microsoft PowerPoint®]) |S2G24C 
Computer-based information resources (e.g., websites, wikis, IS2G24| 
encyclopaedia) 
Recoding 
1 = Never iL O 
2 = In some lessons ws 1 
3 = In most lessons 3 2 
4 = Inevery or almost every lesson 4 3 
Variable Name: S_ SPECLASS 
Description: Use of specialist applications in class 
Procedure: RT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 
Interpretation Higher values indicating more frequent use 
Source: When studying throughout this school year, how often did you use the following tools during class? 
Please mark one choice in each row) 
ultimedia production tools (e.g., media capture and editing, $2G24E 
web production) 
Concept mapping software (e.g., [Inspiration®], [Webspiration®]) S2G24F 
Tools that capture real-world data (e.g., speed, temperature) S$2G24G 
digitally for analysis 
Simulations and modelling software $2G24H 
Interactive digital learning resources (e.g., learning games or $2624) 
applications) 
Graphing or drawing software S2G24 
Recoding 
1= Never 1 O 
2 = Insome lessons 2 1 
3 = In most lessons 3 2 
4 = Inevery or almost every lesson 4 3 
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Variable Name: 


S_ICTLRN 


Description: 


Learning of ICT tasks at school 


Procedure: 


RT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 


Interpretation 


Higher values indicating higher level of knowledge 


Source: 


At school, to what extent have you learned how to do the following tasks? 
Please mark one choice in each row) 


Provide references to Internet sources S2G25A 
Search for information using ICT S$2G25B 
Present information for a given audience or purpose using ICT S2G25C 
Work out whether to trust information from the Internet $2G25D 
Decide what information obtained from the Internet is relevant $2G25E 


o include in school work 


Organize information obtained from Internet sources S$2G25 


Decide where to look for information on the Internet about an S2G25G 
unfamiliar topic 


Use ICT to collaborate with others S2G25H 


Recoding 


= Toa large extent 


Nh 


= To amoderate extent 


= Toasmall extent 
4=Notatall 


BR oO NM FB 
Or 


Variable Name: 


S_GENEFF 


Description: 


CT self-efficacy regarding the use of general applications 


Procedure: 


RT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 


Interpretation 


igher values indicating higher level of self-efficacy 


Source: 


ow well can you do each of these tasks when using ICT? 
Please mark one choice in each row) 


Edit digital photographs or other graphic images $2G27A 
Write or edit text for a school assignment $2G27C 
Search for and find relevant information for a school project $2G27D 
on the Internet 
Create a multi-media presentation (with sound, pictures, or video) S2G27 
Upload text, images, or video to an online profile S$2G275 
Insert an image into a document or message S2G27 
Install a program or [app] §$2G27L 
Judge whether you can trust information you find on the Internet S2G27 
Recoding 
1 =I know how to do this 1 2 
2 =| have never done this but | could work out how to do this 2 1 
3 =| do not think | could do this 3 6) 
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Variable Name: 


SES PEGERE 


Description: 


CT self-efficacy regarding the use of specialist applications 


Procedure: 


RT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 


Interpretation 


igher values indicating higher level of self-efficacy 


Source: ow well can you do each of these tasks when using ICT? 
Please mark one choice in each row) 
Create a database (e.g., using [Microsoft Access®)]) |S2G27B 
Build or edit a webpage IS2G27E 
Create a computer program, macro, or [app] IS2G27G 
e.g., in [Basic, Visual Basic]) 
Set up a local area network of computers or other ICT IS2G27H 
Recoding 
1 =I know how to do this 1 2 
2 =| have never done this but | could work out how to do this 2 i 
3 =| do not think | could do this 3 O 
VariableName: S_ICTPOS 


Description: 


Perceptions of positive outcomes of ICT for society 


Procedure: 


RT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 


Interpretation 


igher values indicating stronger agreement 


Source: 


ow much do you agree or disagree with the following statements about ICT? 
Please mark one choice in each row) 


Recoding 


Advances in ICT usually improve people's living conditions IS2G28A 

CT helps us to understand the world better IS2G28B 

CT is valuable to society IS2G28F 
Advances in ICT bring many social benefits IS2G28G 

1 = Strongly agree 1 3 
2 = Agree 2 2 
3 = Disagree 3 i 
4 = Strongly disagree 4 O 


Variable Name: 


S_ICTNEG 


Description: 


Perceptions of negative outcomes of ICT for society 


Procedure: 


RT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 


Interpretation 


igher values indicating stronger agreement 


Source: 


ow much do you agree or disagree with the following statements about ICT? 
Please mark one choice in each row) 


Recoding 


Using ICT makes people more isolated in society IS2G28C 
With more ICT there will be fewer jobs IS2G28D 
People spend far too much time using ICT IS2G28E 
Using ICT may be dangerous for people's health IS2G28H 
1 = Strongly agree 1 3 
2 = Agree 2 2 
3 = Disagree 3 al 
4 = Strongly disagree 4 O 
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Variable Name: 


S_ICTFUT 


Description: 


Expectations of future ICT use for work and study 


Procedure: 


RT WLE scores with mean of 50 and stand 


ard deviation of 10 for equally weighted countries 


Interpretation 


igher values indicating stronger agreement 


Source: 


am interested in 


1 = Strongly agree 
2 =Agree 
3 = Disagree 


would like to study subjects related to ICT after [secondary school] 


hope to find a job that involves advanced ICT 


Learning how to use ICT applications will help me to do the work 


4 = Strongly disagree 


ow much do you agree or disagree with the following statements about ICT? 
Please mark one choice in each row) 


IS2G28 
IS2G28 
IS2G28 


EB woON BR 


| 
J 
K 


Recoding 


OrRN 


Variable Name: 


S_CODLRN 


Description: 


Learning of ICT coding tasks at school 


Procedure: 


RT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 


Interpretation 


Higher values indicating higher level of knowledge 


Source: 


2=7 


ollowing tasks? 


o break a complex 


To understand diag 


o plan tasks by set 


To use real-world d 


= To alarge exten 


When studying duri 


Please mark one choice in each row) 


To display information in different ways 


process into smaller parts 


rams that describe or show real-world problems 


ting out the steps needed to complete them 


To use tools to make diagrams that help solve problems 
To use simulations to help understand or solve real-world problems 
To make flow diagrams to show the different parts of a process 


To record and evaluate data to understand and solve a problem 


ata to review and revise solutions to problems 


t 


To amoderate extent 
3 = To asmall extent 
4=Notatall 


$2G29 
S$2G29 
$2G29 
$2G29 
$2G29 
$2G29 
$2G29 
$2G29 


BwWoN BR 


S2G29A 


B 
Cc 


G 
H 


ng the current school year, to what extent have you been taught how to do the 


Recoding 


OrRN 
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Section 2: Principal questionnaire 


Simple indices 


Variable Name: 


P_SEX (available in the Restricted Use File) 


Description: Sex of principal 
Procedure: Simple recoding 
Source: Are you female or male? IP2GO1 Recoding 
1= Female 1 1 
2= Male 2 O 
Variable Name: P_NUMSTD (available in the Restricted Use File) 
Description: Number of students in school (School size) 
Procedure: P_NUMSTD = IP2GO03A+IP2G03B 
Source: What is the total number of boys and girls in the school? 
(Please record a whole number. Record O (zero), if none.) 
Total number of girls IP2GO3A 
Total number of boys IP2GO03B 
Variable Name: P_NUMSTD_CAT 
Description: Number of students in school (School size) 
Procedure: Categorize PNUMSTD 
Source: What is the total number of boys and girls in the school? 
(Please record a whole number. Record O (zero), if none.) 
1= 1-300 
2 = 301-600 
3 = 601-900 
4 = more than 900 
Variable Name: P_NUMTAR (available in the Restricted Use File) 
Description: Number of students in target grade 
Procedure: P_NUMTAR = IP2GO4A +IP2G04B 
Source: What is the total number of boys and girls in [target grade]? 
(Please record a whole number. Record O (zero), if none.) 
Total number of girls IP2GO04A 
Total number of boys |IP2G04B 


Variable Name: 


P_NUMTAR_CAT 


Description: Number of students in target grade 
Procedure: Categorize PNUMTAR 
Source: What is the total number of boys and girls in [target grade]? 


(Please record a whole number. Record O (zero), if none.) 
1= 1-100 

2 = 101-200 

3 = more than 200 
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Variable Name: 


P_NGRADE 


Description: Number of grades in school 
Procedure: P_NGRADE = IP2GO05B - IP2GO5A 
Source: What is the lowest (youngest) grade that is taught at your school? IP2GO5A 


(Please mark only once choice) 


1= 
2= 


ati 
ati 
ati 
ati 
ati 
ati 
ati 
ati 


What is 


(Please mark only once choice) 


ati 
ati 
ati 
ati 
ati 
ati 


ona 


ona 


ona 


ona 


ona 


ona 


ona 


ona 


ona 


ona 


ona 


ona 


ona 


ona 


Adaptation 1 
Adaptation 2 
Adaptation 3 
Adaptation 4 
Adaptation 5 
Adaptation 6 
Adaptation 7 
Adaptation 8 


the highest (oldest) grade that is taught at your school? IP2GO5B 


Adaptation 9] 
Adaptation 10] 
Adaptation 11] 
Adaptation 12] 
Adaptation 13] 
Adaptation 14] 


Variable Name: 


P_NUMTCH (available in the Restricted Use File) 


Description: Number of teachers 
Procedure: P_NUMTCH=(IP2GO6A + 0.5* IP2GO6B) 
Source: What are the total numbers of full-time and part-time teachers in your school? 


A full-time teacher is employed on a regular basis as a teacher for at least 90% of full-time hours for the 
full school year. All other teachers should be considered part-time. 


(Please record a whole number. Record O (zero), if none.) 


Total number of full-time teachers IP2GO06A 
Total number of part-time teachers |IP2GO06B 


Variable Name: 


P_NUMTCH_CAT 


Description: Number of teachers 
Procedure: Categorize P NUMTCH 
Source: What are the total numbers of full-time and part-time teachers in your school? 


A full-time teacher is employed on a regular basis as a teacher for at least 90% of full-time hours for the 
full school year. All other teachers should be considered part-time. 


(Please record a whole number. Record O (zero), if none.) 


1=1-25 
2 = 26-50 
3=51-75 


4=morethan 75 
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Variable Name: 


P_RATTCH 


Description: Ratio of school size and teachers 
Procedure: P_RATTCH=P_NUMTCH/P_NUMSTD 
Source: What are the total numbers of full-time and part-time teachers in your school? 


ull school year. All other teachers should be considered part-time. 
Please record a whole number. Record O (zero), if none.) 


Total number of full-time teachers IP2GO6A 
Total number of part-time teachers |IP2G06B 


A full-time teacher is employed on a regular basis as a teacher for at least 90% of full-time hours for the 


Variable Name: 


P_PRIV (available in the Restricted Use File) 


Description: 


Private school indicator 


Procedure: 


Simple recoding 


Source: 


s this school a public or a private school? IP2GO08A 
Please mark only one choice) 


1 =A public school 1 
This is a school managed directly or indirectly by a public education 
authority, government agency, or governing board, appointed by 

government or elected by public franchise) 


2 = Aprivate school 2 
(This is a school managed directly or indirectly by a 

non-government organization; for example, a church, trade union, 

business, or other private institution) 


Recoding 


O 
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Variable Name: 


P_ICTUSE 


Description: 


Principals’ use of ICT for general school-related activities 


Procedure: 


RT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 


Interpretation 


igher values indicating more frequent ICT use 


Source: 


ow often do you use ICT for the following activities? 
Please mark one choice in each row) 


Provide information about an educational issue through a website P2G02B 
Look up records in a database (e.g., ina student information system) P2G02C 

aintain, organize, and analyze data (e.g., with a spreadsheet or P2G02D 
database) 
Prepare presentations P2GO2E 
Work with a learning management system (e.g., [Moodle]) P2G02J 
Use social media to communicate with the wider community about P2G02 
school-related activities 

anagement of staff (e.g., scheduling, professional development) P2GO02L 
Preparing the curriculum P2GO02 
School financial management P2G02 

Recoding 
1 = Never 1 0 
2 = Less than once a month 2 6) 
3 = At least once a month but not every week 3 1 
4 = At least once a week but not every day 4 2 
5 = Every day 5 3 
Variable Name: P_ICTCOM 


Description: 


Principals’ use of ICT for school-related communication activities 


Procedure: 


RT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 


Interpretation 


igher values indicating more frequent use 


Source: 


ow often do you use ICT for the following activities? 
Please mark one choice in each row) 


Communicate with teachers in your school 
Communicate with education authorities 


Communicate with principals and senior staff in other schools 


Communicate with parents 


1=Never 

2 = Less than once a month 

3 = At least once a month but not every week 
4 = At least once a week but not every day 


5 = Every day 


|[P2GO2F 
IP2GO02G 
IP2G02H 
|IP2G02| 


mE WON BR 


Recoding 
0 


O 
1 
2 
3 
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Variable Name: P_VWICT 


Description: View on using ICT for educational outcomes 
Procedure: RT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 
Interpretation igher values indicating higher importance 
Source: ow important is each of the following outcomes of education in your school? 
Please mark one choice in each row) 
The development of students’ basic computer skills P2GO09A 
e.g., internet use, email, word processing, presentation software) 
The development of students’ skills in using ICT for collaboration P2GO9B 
with others 
The use of ICT for facilitating students’ responsibility for their P2GO09C 


own learning 


The use of ICT to augment and improve students’ learning P2GO9D 
The development of students’ understanding and skills relating to P2GO9E 
safe and appropriate use of ICT 
The development of students’ proficiency in accessing and using P2GO9F 
information with ICT 
Recoding 
| = Very important 4 3 
2 = Quite important 2 2 
3 = Somewhat important 3 ih 
4 = Not important 4 O 
Variable Name: P_EXPLRN 
Description: CT use expected of teachers 
Procedure: RT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 
Interpretation Higher values indicating higher level of expectation 
Source: Are teachers in your school expected to acquire knowledge and skills in each of the following activities? 
Please mark one choice in each row) 
ntegrate Web-based learning in their instructional practice P2G11A 
Use ICT-based forms of student assessment P2G11B 
Use ICT for monitoring student progress P2G11C 
ntegrate ICT into teaching and learning P2G11G 
Use subject-specific digital learning resources (e.g., tutorials, P2G11H 
simulation) 
Use e-portfolios for assessment P2G11I 
Use ICT to develop authentic (real-life) assignments for students P2G11J 
Assess students’ [computer and information literacy] P2G11K 
Recoding 
1 = Expected and required 1 2 
2 = Expected but not required 2 fl 


3 = Not expected 3 O 
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Variable Name: 


P_EXPTCH 


Description: 


Expectations for 


teacher collaboration using ICT 


Procedure: 


IRT WLE scores wit 


h mean of 50 and standard deviation of 10 for equally weighted countries 


Interpretation 


Higher values indicating higher level of expectation 


Source: 


Are teachers in you 


(Please mark one choice in each row) 


Collaborate with other teachers via ICT 


Communicate with parents via ICT 


Communicate with students via ICT 


1 = Expected and required 


2 = Expected but not required 


3 = Not expected 


r school expected to acquire knowledge and skills in each of the following activities? 


IP2G11D 
IP2G11E 
IP2G11F 
Recoding 
2 
1 
0 


Variable Name: 


P_PRIORH 


Description: 


Priorities of increased ICT resources 


Procedure: 


RT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 


Interpretation 


Higher values indicating higher priority 


Source: 


earning? 


ncreasing 


ncreasing 


he nu 


Please mark one choice in each row) 


bers of computers per student in the school 


he number of computers connected to the Internet 


ncreasing the bandwidth of Internet access for the computers 


connected to the Internet 


1 = High priority 


2 = Medium priority 


3 = Low priority 


4 = Nota priority 


At your school, what priority is given to the following ways of facilitating the use of ICT in teaching and 


IP2G15A 
IP2G15B 
IP2G15C 
Recoding 
3 
2 
1 
0 
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Variable Name: P_PRIORS 


Description: Priorities of increased ICT professional learning resources 
Procedure: RT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 
Interpretation Higher values indicating higher priority 
Source: At your school, what priority is given to the following ways of facilitating the use of ICT in teaching and 

earning? 

Please mark one choice in each row) 

ncreasing the range of digital learning resources available for P2G15D 

eaching and learning 

Establishing or enhancing an online learning support platform P2G15E 

Supporting participation in professional development on P2G15F 


pedagogical use of ICT 


ncreasing the availability of qualified technical personnel to support IP2G15G 


he use of ICT 

Providing teachers with incentives to integrate ICT use in their P2G15H 
eaching 

Providing more time for teachers to prepare lessons in which ICT P2G15l 
is used 


ncreasing the professional learning resources for teachers inthe use IP2G15J 


of ICT 
Recoding 
1 = High priority 1 3 
2 = Medium priority 2 2 
3 = Low priority 3 i 
4 = Nota priority 4 O 
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Variable Name: 


C_EXP 


Description: ICT experience in years in the school 
Procedure: Simple recoding 
Source: How many years has your school been using ICT for teaching and/or 112GO03 Recoding 
learning purposes for students in [target grade]? 
(Please mark one choice) 
1 = Never, we do not use ICT 1 O 
2 = Fewer than 5 years 2 1 
3 = At least 5 but fewer than 10 years 3 2 
4= 10 years or more 4 3 


Variable Name: C_ICTDEV 
Description: Sum of ICT devices 
Procedure: C_ICTDEV=SUM(AA1,AB1,AC1) 
Source: In your school, approximately how many of the following types of (school-provided) ICT devices are 


available? 


(Please record a whole number. Record O (zero), if none.) 


For this question 
e¢ Count termina 


please: 
s (if they have a keyboard and a screen) as computers 


e Exclude computers that are not in use (e.g., in storage) 


e Exclude compu 


e Record 0 (zero 


Laptops/noteboo 


ters that are only used as servers 


), if none. 
Desktop computers - All devices in the school I2GO7AA1 
s - All devices in the school I2GO7AB1 
| devices in the school I2GO7AC1 


Tablet devices - A 


Variable Name: 


C_RATDEV 


Description: Ratio of school size and number of ICT devices 
Procedure: C_RATDEV=P_NUMSTD/C_ICTDEV 
Source: In your school, approximately how many of the following types of (school-provided) ICT devices are 


available? 


(Please record a whole number. Record O (zero), if none.) 


For this question 
¢ Count termina 
e Exclude compu 
e Exclude compu 


please: 

s (if they have a keyboard and a screen) as computers 
ters that are not in use (e.g., in storage) 

ers that are only used as servers 


e Record 0 (zero), if none. 
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Variable Name: C_ICTSTD 


Description: Sum of ICT devices available for student use 

Procedure: C_ICTSTD=SUM(AA2,AB2,AC2) 

Source: In your school, approximately how many of the following types of (school-provided) ICT devices are 
available? 


(Please record a whole number. Record O (zero), if none.) 


For this question please: 

e Count terminals (if they have a keyboard and a screen) as computers 
e Exclude computers that are not in use (e.g., in storage) 

e Exclude computers that are only used as servers 

e Record 0 (zero), if none. 


Desktop computers - Devices available for student use I2GO7AA2 
Laptops/notebooks - Devices available for student use I12GO07AB2 
Tablet devices - Devices available for student use I2GO7AC2 


Variable Name: C_RATSTD 


Description: Ratio of school size and number of devices available for students 

Procedure: C_RATSTD=P_NUMSTD/C_ICTSTD 

Source: In your school, approximately how many of the following types of (school-provided) ICT devices are 
available? 


(Please record a whole number. Record O (zero), if none.) 


For this question please: 
e Count terminals (if they have a keyboard and a screen) as computers 
e Exclude computers that are not in use (e.g., in storage) 

e Exclude computers that are only used as servers 

e Record 0 (zero), if none. 


Variable Name: C_RATSMB 


Description: Ratio of school size and smart boards 
Procedure: C_RATSMB=P_NUMSTD/II2GO7C 
Source: In your school, about how many (school-provided) smart boards or interactive whiteboards are available? 


(Please record a whole number. Record O (zero), if none.) 


APPENDIX C 


Scale indices 


253 


Variable Name: 


GsIGTIRES 


Description: 


Availability of ICT 


resources at school 


Procedure: 


IRT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 


Interpretation 


Higher values indi 


cating higher availability 


Source: 


Please indicate the availability of each of the following software resources at your school 


(Please mark one choice in each row) 


Digital learning resources that can only be used online 
Access to the Internet through the school network II2GO04C 


Access to an education site or network maintained by education 


authorities 


Email accounts fo 


r school-related use 


Practice programs or [apps] where teachers decide which questions 


are asked of students (e.g., [Quizlet, Kahoot], [mathfessor 


Single user digital 


Multi-user digital 
(e.g., [Quest Atlan 


Video and photo software for capture and editing 


(e.g., [Windows 


) 
learning games (e.g., [languages online]) 


learning games with graphics and inquiry tasks 
tis]) 


ovie Maker, iMovie, Adobe Photoshop]) 


Concept mapping software (e.g., [Inspiration®], [Webspiration®]) 


Data logging and 
real-world data di 


monitoring tools (e.g., [Logger Pro]) that capture 


gitally for analysis (e.g., speed, temperature) 


A learning management system (e.g., [Edmodo], [Blackboard]) 


e-portfolios (e.g., 


VoiceThread]) 


Digital contents li 


nked with textbooks 


1 = Available to teachers and students 


2 = Available only 
3 = Available only 
4 = Not available 


to teachers 


to students 


BwoON HE 


2G04 


2G04 


2G04 


2G05A 


2G05 
2G05 


2G05 


2G05 
2G05 


2G055 


2G05 
2G05 


B 
GC 


G 
H 


Recoding 


ORF 
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Variable Name: C_HINRES 


Description: Computer resource hindrances to the use of ICT in teaching and learning 
Procedure: RT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 
Interpretation Higher values indicating fewer hindrances 
Source: To what extent is the use of ICT in teaching and learning at your school hindered by each of the following 
obstacles? 
Please mark one choice in each row) 
Too few computers with an Internet connection 2G13A 
nsufficient Internet bandwidth or speed 2G613B 
ot enough computers for instruction 2G613C 
Lack of sufficiently powerful computers 2G13D 
Problems in maintaining ICT equipment 2G13E 
ot enough computer software 2G13F 
Recoding 
1=Alot 1 3 
2 = To some extent 2 
3 = Very little 3 1 
4=Notatall 0 
Variable Name: C_HINPED 
Description: Pedagogical resource hindrances to the use of ICT in teaching and learning 
Procedure: RT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 
Interpretation Higher values indicating fewer hindrances 
Source: To what extent is the use of ICT in teaching and learning at your school hindered by each of the following 
obstacles? 
Please mark one choice in each row) 
nsufficient ICT skills among teachers 2G13G 
nsufficient time for teachers to prepare lessons 2G13H 
Lack of effective professional learning resources for teachers 26131 
Lack of an effective online learning support platform 26135 
Lack of incentives for teachers to integrate ICT useintheir teaching Il2G13K 
nsufficient pedagogical support for the use of ICT 2G13N 
Recoding 
1=Alot 1 3 
2 = Tosome extent 2 2 
3 = Very little 3 il 
4=Notatall 4 O 
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Variable Name: T_SEX 
Description: Sex of teacher 
Procedure: Simple recoding 
Source: Are you female or male? IT2GO1 Recoding 
1 = Female 1 1 
2=Male 2 0 
Variable Name: T_AGE 
Description: Approximate age of teacher 
Procedure: Simple recoding 
Source: How old are you? IT1G02 Recoding 
(Please mark only one choice) 
1=Less than 25 1 23 
2= 25-29 2 oh 
3 = 30-39 3 35 
4= 40-49 4 45 
5 = 50-59 5 55 
6 = 60 or over 6 63 
Variable Name: T_WGT 
Description: Allocation of teacher’s staff time to sampled school 
Procedure: Simple recoding 
Source: Inthe current school year, at how many schools do you teach IT2G04 Recoding 
[target grade] students? 
(Please mark only one choice) 
1 = Only in this school al 1,00 
2 = Inthis and another school 2 0,50 
3 = Inthis and in two other schools 3 0,33 
4 = Inthis and in three or more other schools 4 0,25 
Variable Name: T_EXLES 
Description: ICT experience with ICT use during lessons 
Procedure: Simple recoding 
Source: Approximately how long have you been using ICT for teaching purposes? Recoding 
(Please mark only one choice in each row) 
During lessons IT2GO5A 
1 = Never i: O 
2 = Less than two years 2 i 
3 = Between two and five years 3 2 
4 = More than five years 4 3 
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Variable Name: 


T_EXPREP 


Description: ICT experience with ICT use for preparing lessons 
Procedure: Simple recoding 
Source: Approximately how long have you been using ICT for teaching purposes? 


(Please mark only one choice in each row) 
Preparing lessons 

1 = Never 

2 = Less than two years 

3 = Between two and five years 


4 = More than five years 


IT2GO5B 
1 


2 
3 
4 


Recoding 
O 


1 
2 
3 


Scale indices 


Variable Name: 


DelGiERR 


Description: 


Teachers ICT self-efficacy 


Procedure: 


RT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 


Interpretation 


igher values indicating higher level of self-efficacy 


Source: 


ow well can you do these tasks using ICT? 
Please mark one choice in each row) 


Find useful teaching resources on the Internet 


Contribute to a discussion forum/user group on the Internet 
e.g., a wiki or blog) 


Produce presentations (e.g., [PowerPoint® or a similar program]), 
with simple animation functions 


Use the Internet for online purchases and payments 


Prepare lessons that involve the use of ICT by students 


Using a spreadsheet program (e.g., [Microsoft Excel®]) for keeping 
records or analyzing data 


Assess student learning 


Collaborate with others using shared resources such as 
Google Docs®], [Padlet] 


Use a learning management system (e.g., [Moodle], [Blackboard], 
Edmodo]) 


1 =| know how to do this 
2 =| haven't done this but | could find out how 
3 = 1 do not think | could do this 


T2GO7A 
T2G07B 


T2G07C 


T2G07D 


T2GO7E 
T2GO07 


T2GO7G 
T2G07H 


IT2GO7| 


Recoding 
2 


ul 
O 
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Variable Name: 


T_ICTEMP 


Description: 


Emphasis on ICT capabilities in class 


Procedure: 


RT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 


Interpretation 


Higher values indicating stronger emphasis 


Source: 


he following I|CT-based capabilities in your students? 


Please mark one choice in each row) 

To access information efficiently 

To display information for a given audience/purpose 
To evaluate the credibility of digital information 

To share digital information with others 


To use computer software to construct digital work products 
(e.g., presentations, documents, images and diagrams) 


To provide digital feedback on the work of others (such 
as classmates) 


To explore a range of digital resources when searching for 
information 


To provide references for digital information sources 


To understand the consequences of making information publically 
available online 


| = Strong emphasis 
2 = Some emphasis 
3 = Little emphasis 


4 = No emphasis 


T2GO9A 
T2GO09B 


T2GO09D 
T2GO9E 


T2GO9F 


T2GO09G 


BwWON BR 


Recoding 
3 


2 
1 
O 


nyour teaching the reference class in this school year, how much emphasis have you given to developing 
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Variable Name: 


T_CLASACT 


Description: 


Use of ICT for clas 


sroom activities 


Procedure: 


RT WLE scores wi 


th mean of 50 and standard deviation of 10 for equally weighted countries 


Interpretation 


igher values indicating more frequent use 


Source: 


ow often do stud 


Please mark one choice in each row) 


Work on extended projects (i.e., lasting over a week) T2G1 
Work on short assignments (i.e., within one week) T2G1 
Explain and discuss ideas with other students T2G1 
Submit completed work for assessment T2G1 
Work individually on learning materials at their own pace T2G1 
Undertake open-ended investigations or field work T2G1 
Reflect on their learning experiences (e.g., by using a learning log) T2G1 
Communicate with students in other schools on projects T2G1 
Plan a sequence of learning activities for themselves T2G1 
Analyze data T2G1 
Evaluate information resulting from a search T2G1 
Collect data for a project T2G1 
Create visual products or videos T2G1 
Share products with other students T2G1 
1 = They do not engage in this activity 1 

2 = They never use ICT in this activity 2 

3 = They sometimes use ICT in this activity 3 

4 = They often use ICT in this activity 4 

5 = They always use ICT in this activity 5 


ents in your reference class use ICT for the following activi 


m Cj 


@) 


Oo oO OO Oo © OG © 2 02 02 9 2 6 
= =E 


Recoding 
Missing 
0 


f 
2 
3 
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Variable Name: 


T_ICTPRAC 


Description: 


Use of ICT for teaching practices in class 


Procedure: 


RT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 


Interpretation 


igher values indicating more frequent use 


Source: 


Please mark one choice in each row) 


The provision of remedial or enrichment support to individual 
tudents or small groups of students 


Yn 


The support of student-led whole-class discussions and 
presentations 


he assessment of students’ learning through tests 


[he provision of feedback to students on their work 


The support of collaboration among students 


external mentors 


The support of inquiry learning 


| =| donot use this practice with the reference class 
2 =| never use ICT with this practice 

3 =| sometimes use ICT with this practice 

4 =| often use ICT with this practice 


5 =| always use ICT with this practice 


[he reinforcement of learning of skills through repetition of examples 


The mediation of communication between students and experts or 


T2611 


T2G1 


T2G1 
T2G1 
T2G1 
T2G1 
T2G1 


T2G1 


mam BR WON PB 


ow often do you use ICT in the following practices when teaching your reference class? 


= 


Recoding 
Missing 
0 


1 
Z 
3 


Variable Name: 


T_USETOOL 


Description: 


Use of digital learning tools 


Procedure: 


RT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 


Interpretation 


igher values indicating more frequent use 


Source: 


Please mark one choice in each row) 


Practice programs or apps where you ask students questions 
e.g., [Quizlet, Kahoot], [mathfessor]) 


Digital learning games 


Concept mapping software (e.g., [Inspiration®], [Webspiration®]) 
Simulations and modelling software (e.g., [NetLogo]) 

A learning management system (e.g., [Edmodo], [Blackboard]) 
Collaborative software (e.g., [Google Docs®], [Onenote]) [Padlet]) 
Interactive digital learning resources (e.g., learning objects) 


Graphing or drawing software 


e-portfolios (e.g., [VoiceThread]) 


Social media (e.g., [Facebook, Twitter]) 


1 = Never 


nsome lessons 


oO 
iT 


n most lessons 


4 =Inevery or almost every lesson 


ow often did you use the following tools in your teaching of the reference c 


T2G1 


T2G1 
T2G1 
T2G1 
[2G1 
T2G1 
T2G1 
[2G1 
[2G1 


BwWON BR 


[2G1 


ass this school year? 


2A 


108) 


ae 


Recoding 
O 


1 
2 
3 
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Variable Name: T_USEUTIL 


Description: Use of general utility software 

Procedure: RT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 
Interpretation igher values indicating more frequent use 

Source: ow often did you use the following tools in your teaching of the reference class this school year? 


Please mark one choice in each row) 
Word-processor software (e.g., [Microsoft Word®]) IT2G12C 


Presentation software (e.g., [Microsoft PowerPoint®)]) IT2G12D 


Computer-based information resources (e.g., topic-related websites, IT2G12L 


wikis, encyclopaedia) 
Digital contents linked with textbooks IT2G12P 
Recoding 
= Never 1 O 
2 = In some lessons 2 al 
3 = In most lessons 3 2 
= In every or almost every lesson 4 3 
Variable Name: T_CODEMP 
Description: Teacher emphasis of teaching coding tasks in class 
Procedure: IRT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 
Interpretation Higher values indicating higher emphasis 
Source: In your teaching of the reference class this school year, how much emphasis have you given to teaching the 
following skills? 
(Please mark one choice in each row) 
To display information in different ways T2G13A 
To break a complex process into smaller parts IT2G13B 
To understand diagrams that describe or show real-world problems 1T2G13C 
To plan tasks by setting out the steps needed to complete them T2G13D 
To use tools making diagrams that help solve problems T2G13E 
To use simulations to help understand or solve real-world problems = 1T2G13F 
To make flow diagrams to show the different parts of a process T2G13G 
To record and evaluate data to understand and solve a problem T2G13H 
To use real-world data to review and revise solutions to problems T2G13 
Recoding 
| = Strong emphasis 1 3 
2 = Some emphasis 2 2 
3 = Little emphasis 3 4 
4 = No emphasis 4 O 
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Variable Name: T_RESRC 


Description: Availability of computer resources at school 
Procedure: IRT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 
Interpretation Higher values indicating stronger agreement 
Source: To what extent do you agree or disagree with the following statements about using ICT in teaching at your 
school? 
Please mark one choice in each row) 
y school has sufficient ICT equipment (e.g., computers) T2G14B 
The computer equipment in our school is up-to-date T2G14C 
y school has access to sufficient digital learning resources T2G14D 


e.g., learning software or [apps]) 


y school has good connectivity (e.g., fast speed - same asin STable) IT2G14E 
o the Internet 


There is enough time to prepare lessons that incorporate ICT T2G14F 
There is sufficient opportunity for me to develop expertise in ICT T2G14G 
There is sufficient technical support to maintain ICT resources T2G14H 
Recoding 
| = Strongly agree 1 
2 =Agree 2 2 
3 = Disagree 3 1 
4 = Strongly disagree 4 O 
Variable Name: T_COLICT 
Description: Collaboration between teachers in using ICT 
Procedure: IRT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 
Interpretation Higher values indicating stronger agreement 
Source: To what extent do you agree or disagree with the following statements about your use of ICT in teaching 
and learning at your school? 
Please mark one choice in each row) 
work together with other teachers on improving the use of ICT T2G15A 
in classroom teaching 
collaborate with colleagues to develop ICT-based lessons T2G15B 
observe how other teachers use ICT in teaching T2GI5C 
discuss with other teachers how to use ICT in teaching topics T2G15D 
share |CT-based resources with other teachers in my school T2G15E 
Recoding 
1 = Strongly agree 41 3 
2 =Agree 2 2 
3 = Disagree 3 1 
4 = Strongly disagree 4 0 
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Variable Name: 


T_PROFSTR 


Description: 


Teacher participation in structured learning professional development related to ICT 


Procedure: 


RT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 


Interpretation 


igher values indicating more frequent participation 


Source: 


ow often have you participated in any of the following professional learning activities in the past two 
years? 


Please mark one choice in each row) 


Acourse on ICT applications (e.g., word processing, presentations, T2G17A 

internet use, spreadsheets, databases) 

A course or webinar on integrating ICT into teaching and learning T2G17B 

Training on subject-specific digital teaching and learning resources T2G17C 

A course on use of ICT for [students with special needs or specific T2G17H 

learning difficulties] 

A course on how to use ICT to support personalized learning by T2G17) 

students 

Recoding 

1 =Notat all 1 O 

2 = Once only 2 fl 

3 = More than once S 2 
Variable Name: T_PROFREC 


Description: 


Teacher participation in reciprocal learning professional development related to ICT 


Procedure: 


IRT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 


Interpretation 


Higher values indicating more frequent participation 


Source: 


“How often have you participated in any of the following professional learning activities in the past two 
years?” 


(Please mark one choice in each row) 


Observations of other teachers using ICT in teaching IT2G17D 

An |CT-mediated discussion or forum on teaching and learning IT2G17E 

The sharing of digital teaching and learning resources with others IT2G17F 

through a collaborative workspace 

Use of a collaborative workspace to jointly evaluate student work IT2G17G 

Recoding 

1=Notatall 1 O 

2 = Once only 2 cl 


3 = More than once 3 2 
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Variable Name: T_VWNEG 
Description: Negative views on using ICT in teaching and learning 
Procedure: IRT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 


Interpretation 


Higher values indicating stronger agreement 


Source: 


To what extent do you agree or disagree with the following practices and principles in relation to the use of 


mpe 
Resu 
Distr 
Resu 


Resu 


Limit 


1=S 


CT in teaching and learning? 
Please mark one choice in each row) 
Using ICT at school: 


des concept formation by students 

ts in students copying material from Internet sources 

acts students from learning 

ts in poorer written expression among students 

ts in poorer calculation and estimation skills among students 


s the amount of personal communication among students 


trongly agree 


2 = Agree 


3 = Disagree 


4 = Strongly disagree 


BwWN BR 


[2G1 
T2G1 
[2G1 
[2G1 
[2G1 
[2G1 


8A 
8D 
8F 
8G 
8H 
8l 


Recoding 


OrRN 


Variable Name: 


T_VWPOS 


Description: 


Positive views on using ICT in teaching and learning 


Procedure: 


IRT WLE scores with mean of 50 and standard deviation of 10 for equally weighted countries 


Interpretation 


Higher values indicating stronger agreement 


Source: 


To what extent do you agree or disagree with the following practices and principles in relation to the use of 


nab 


heir 


ab 


CT in teaching and learning? 

Please mark one choice in each row) 

Using ICT at school: 

elps students develop greater interest in learning 
elps students to work at a level appropriate to their learning needs 


elps students develop problem-solving skills 


les students to collaborate more effectively 


Helps students develop skills in planning and self-regulation of 


work 


mproves academic performance of students 


les students to access better sources of information 


1 = Strongly agree 


2=A 


gree 


3 = Disagree 


4 = Strongly disagree 


T2G1 
T2G1 
[2G1 
[2G1 
2G1 


[2G1 


BwWON BR 


[2G1 


8B 
8C 
8E 
8 

8K 


8L 
8M 


Recoding 


OrRN 


IEA’s International Computer and Information Literacy Study (ICILS) 2018 is designed 
to assess how well students are prepared for study, work, and life in a digital world. 
The study measures international differences in students’ computer and information 
literacy (CIL): their ability to use computers to investigate, create, participate, and 
communicate at home, at school, in the workplace, and in the community. Participating 
countries also have an option for their students to complete an assessment of 
Ceroyanyolele-la(oyar-) manlials<iayca (Oa B 


The ICILS 2018 user guide describes the content and format of the data in the ICILS 
2018 international database. It introduces the use of weighting and variance estimation 
variables for analyzing the ICILS 2018 data, and provides a comprehensive overview 
of how to work with IEA’s International Database (IDB) Analyzer software. The ICILS 
2018 user guide is accompanied by three appendices: the international versions of all 
questionnaires; an overview of national adaptations to the national versions of the 
Ket | Bsp~Xo ui sMTainclaat-lucolate] mel e(esin(oalar-linetsrar-l ale m= Moo) onl e)ae) eleva tcyh\{eMor=] i=] (os-40(-Mo) ma p\eMe(=lanv{cVe| 
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